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OVERSIGHT  REVIEW  OF  RESEARCH  LABORA- 
TORY PROGRAMS  AT  THE  NATIONAL  INSTI- 
TUTE  OF  STANDARDS  AND  TECHNOLOGY 


TUESDAY,  JUNE  25,  1996 

U.S.  House  of  Representatives, 

Committee  on  Science, 
Subcommittee  on  Technology, 

Washington,  DC. 

The  Subcommittee  met,  pursuant  to  notice,  at  1:11  p.m.,  in  Room 
2325,  Rayburn  House  Office  Building,  Hon.  Constance  A.  Morella, 
Chairwoman  of  the  Subcommittee,  presiding. 

Mrs.  Morella.  I  am  going  to  call  to  order  the  Technology  Sub- 
committee of  the  Science  Committee  for  this  hearing  that  we  have 
on  research  laboratory  programs  at  the  National  Institute  of  Stand- 
ards and  Technology.  I  appreciate  the  willingness  of  panel  number 
two  to  become  panel  number  one.  That  just  shows  your  adapt- 
ability, and  I  appreciate  that. 

Today's  hearing  is  the  third  and  the  final  one  in  a  series  of  hear- 
ings to  review  the  intramural  laboratories  at  the  National  Institute 
of  Standards  and  Technology. 

As  the  Council  on  Competitiveness  noted  in  its  recent  report  en- 
titled, "Endless  Frontier,  Limited  Resources",  the  United  States  re- 
search and  development  enterprise  finds  itself  in  a  wrenching  pe- 
riod of  change  with  the  end  of  the  Cold  War,  the  globalization  of 
the  world  economy,  and  the  drive  to  eliminate  the  Federal  deficit. 
This  environment  creates  both  an  unprecedented  opportunity  and 
a  significant  challenge  for  NIST. 

On  the  one  hand,  NIST  is  the  lead  Federal  laboratory  in  interact- 
ing with  industry,  having  the  statutory  mission  to  promote  U.S. 
economic  growth,  by  working  with  industry  to  develop  and  apply 
technology,  measurements,  and  standards.  Further,  as  the  report 
noted,  research  and  development  partnerships  are  the  key  to  meet- 
ing this  challenge.  However,  Federal  institutions  are  increasingly 
challenged  to  keep  pace  with  technological  change  and  global  mar- 
ket forces,  particularly  in  an  era  of  shrinking  government  budgets. 

So  today's  hearing  will  focus  on  the  Information  Technology  Lab- 
oratory and  the  Electronics  and  Electrical  Engineering  Laboratory. 
In  many  respects,  these  two  laboratories  support  industrial  sectors 
that  are  at  the  forefront  of  our  nation's  technological  revolution  and 
are  defining  our  global  economic  competitiveness. 

The  information  industry  annually  invests  over  $30  billion  in  re- 
search and  development.  Seventy-eight  percent  of  computer  indus- 
try revenues  comes  from  products  that  have  been  on  the  market  for 
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less  than  two  years.  In  recognition  of  the  need  to  focus  on  this 
large,  fast-moving  industry,  NIST  is  proposing  to  merge  two  exist- 
ing laboratories  to  form  the  Information  Technology  Laboratory. 

Likewise,  the  electronics  industry  is  growing  rapidly  and  cur- 
rently accounts  for  nearly  11  percent  of  U.S.  GDP.  For  example, 
the  domestic  semiconductor  industry  is  recovering  from  a  slump  in 
the  mid-1980s  surpassing  the  Japanese  share  of  the  global  semi- 
conductor market  in  1993  for  the  first  time  in  eight  years.  This 
transformation  has  been  aided  by  the  semiconductor  industry's  de- 
velopment of  a  Technology  Roadmap  to  more  efficiently  utilize  the 
nation's  technology  resources,  including  NIST. 

We  are  fortunate  to  have  with  us  today,  on  what  will  then  be- 
come the  second  panel,  the  Directors  of  these  two  NIST  labora- 
tories to  describe  how  their  laboratories  are  responding  to  the  chal- 
lenges. We  also  have  the  Chairs  of  the  National  Research  Council 
panels  that  perform  the  annual  detailed  assessment  of  the  intra- 
mural NIST  laboratory  programs.  Also,  we  are  pleased  to  have  rep- 
resentatives of  NIST's  important  customers  in  industry  to  give 
their  perspectives  on  the  value  of  the  NIST  activities. 

So  due  to  our  flight  delay  for  Dr.  Rhyne  on  the  Electronics  and 
Electrical  Engineering  Laboratory  panel,  we  are  going  to  hear  first 
from  the  Information  Technology  Laboratory  panel.  I  look  forward 
to  the  testimony  of  the  witnesses  and  I  am  now  happy  to  recognize 
the  distinguished  ranking  member  of  the  Subcommittee,  Mr.  Tan- 
ner. 

Mr.  Tanner.  Thank  you.  Madam  Chairwoman,  and  I'll  be  very 
brief. 

I  want  to  welcome  you  all.  This  is  the  third  hearing  on  the  NIST 
laboratory  programs  and  I  was  glad  to  see  the  chair  reference  the 
Council  on  Competitiveness  report,  which  strongly  recommended 
that  the  old  distinction  between  basic  and  applied  research  be  dis- 
pensed with.  I  might  also  add  it  recommended  for  our  consideration 
the  elements  of  the  ATP  and  EMIPRI  programs,  as  well,  but  that 
was  not  to  be. 

Anyway,  NIST's  role  is  to  remove  measurement  barriers  that  in- 
dividual companies  cannot  overcome  because  they  lack  one  or  more 
of  NIST's  special  characteristics  in  partial  position,  recognized 
measurement  capability,  direct  access  to  national  standards,  or  au- 
thority as  the  lead  U.S.  agency  for  measurements.  These  character- 
istics permit  NIST  to  do  what  individual  companies  cannot.  Gain 
access  to  industry  information  needed  to  resolve  industry-wide 
measurement  problems,  develop  measurement  solutions  and  na- 
tional reference  standards  that  benefit  industry  as  a  whole  and 
gain  acceptance  of  those  solutions  by  U.S.  industry  and  its  inter- 
national customers  so  that  U.S.  products  can  be  sold  worldwide. 

I  continue  to  believe  that  NIST  is  the  stealth  agency  of  the  Fed- 
eral Government,  little  understood,  yet  one  of  the  linchpins  of  U.S. 
economic  strength  and  competitiveness.  I  believe  that.  That  is  why 
I'm  on  this  Committee.  I  want  to  take  this  opportunity  to  commend 
the  NIST  employees  who  have  labored  under  some  stress  the  last 
year  or  so,  and  I  want  to  say  once  again  that  the  work  that  you 
all  do  is  very  important  to  this  country  and  it's  important  to  what 
we  do  as  a  country  in  terms  of  economic  industry  growth  and  com- 


petitiveness  in  the  world  economy.  Thank  you,  witnesses,  for  being 
here  today. 

Thank  you. 

Mrs.  MORELLA.  Thank  you,  Mr.  Tanner. 

We  will  start  off  and  hear  from  each  of  you,  and  I  think  we'll  do 
it,  if  there's  no  objection,  in  the  order  from  your  left  to  right.  So 
we'll  start  off  with  Dr.  Shukri  Wakid,  who  is  the  Acting  Director 
of  the  Information  Technology  Laboratory  at  the  National  Institute 
of  Standards  and  Technology  in  Gaithersburg.  Then  we  will  hear 
from  Dr.  Ralph  Roskies,  who  is  Chair  of  the  National  Research 
Council  Board  on  Assessment  of  NIST  Programs  Panel  for  Informa- 
tion Technology.  Then  we  will  hear  from  Dr.  Charles  Brownstein, 
Executive  Director,  Cross-Industry  Working  Team,  Corporation  for 
National  Research  Initiatives,  out  of  Reston,  Virginia. 

Dr.  Wakid,  we  will  start  with  you. 

STATEMENTS  OF  DR.  SHUKRI  A.  WAKID,  ACTING  DIRECTOR, 
INFORMATION  TECHNOLOGY  LABORATORY,  NATIONAL  IN- 
STITUTE OF  STANDARDS  AND  TECHNOLOGY, 
GAITHERSBURG,  MARYLAND;  DR.  RALPH  Z.  ROSKIES,  CHAIR, 
NATIONAL  RESEARCH  COUNCIL  BOARD  ON  ASSESSMENT  OF 
NIST  PROGRAMS  PANEL  FOR  INFORMATION  TECHNOLOGY, 
WASHINGTON,  D.C.;  AND  DR.  CHARLES  N.  BROWNSTEIN,  EX- 
ECUTIVE DIRECTOR,  CROSS-INDUSTRY  WORKING  TEAM, 
CORPORATION  FOR  NATIONAL  RESEARCH  INITIATIVES, 
RESTON,  VIRGINIA 

Dr.  Wakid.  Chairwoman  Morella  and  Members  of  the  Sub- 
committee, thank  you  for  allowing  me  to  speak.  I  would  like  to 
briefly  describe  the  key  role  that  the  NIST  laboratories  play  in  pro- 
viding U.S.  industry  needs  for  measurement  and  standards. 

The  Computer  Systems  Laboratory  and  the  Computing  and  Ap- 
plied Mathematics  Laboratory  provide  the  technical  underpinnings 
and  enabling  tools  that  accelerate  getting  research  into  the  market- 
place and  help  everyone  do  their  job  better. 

Information  technology  is  an  enabling  technology  that  is  becom- 
ing an  essential  tool  for  doing  business  and  is  changing  the  way  we 
do  business.  We  need  to  step  up  to  this  new  opportunity.  To  be 
competitive,  it's  essential  that  we  ease  the  barriers  that  keep  infor- 
mation technology  from  being  a  universal  tool  for  business. 

There  are  barriers  to  the  usability  of  the  technology.  There  are 
barriers  to  the  interoperability  of  the  technology  across  different 
vendors'  systems,  barriers  to  the  scalability  of  the  technology  to 
allow  widespread  access,  and  there  are  barriers  to  the  security  of 
the  technology  to  make  it  dependable  for  business  transaction. 

We  develop  test  methods  that  small  and  large  compainies  need 
and  that  the  users  require.  These  tests  expose  and  ease  the  bar- 
riers that  I  mentioned.  In  many  cases,  industry,  recognizing  our 
neutrality  and  strong  technical  programs,  requests  our  involvement 
and  leadership  in  test  development. 

As  an  example,  imagine  how  useful  it  would  be  for  us  to  talk  to 
computers  and  get  relevant  information.  We  provide  researchers 
with  the  open  pre-competitive  tests  that  allow  the  objective  com- 
parison of  different  approaches  and  enable  the  feasibility  of  apply- 


ing  such  research.  We  develop  tests  such  as  this  for  human-ma- 
chine interfaces,  of  which  speech  is  an  example. 

We  also  develop  tests  for  software  diagnostics  such  that  the  qual- 
ity of  U.S.  software  is  reliable.  In  critical  safety  systems,  software 
must  be  dependable  so  that  the  information  technology  can  be  used 
with  assurance. 

We  develop  tests  for  conformance  to  standards  to  capture  the 
technical  description  of  the  standards  and  ensure  compliance  and 
consistency  in  conforming  to  these  standards.  ISDN,  Integrated 
Services  Digital  Network  Technology,  is  an  example  for  which  we 
developed  conformance  tests. 

We  also  provide  services  to  NIST  scientists,  enabling  them  to  use 
applied  mathematics  and  statistics  and  to  explain  to  the  engineer- 
ing and  scientific  community  how  to  apply  these  underpinning 
methods  to  predict  and  expand  their  industry  to  new  domains.  This 
means  that  they  do  not  have  to  build  very  expensive  prototypes  to 
understand  the  behavior  of  these  new  technologies. 

Finally,  we  have  a  Federal  role  that  is  mandated  by  Congress. 
We  help  Federal  agencies  with  the  use  and  implementation  of 
standards  as  well  as  with  promoting  security  mechanisms  for  un- 
classified and  sensitive  government  information  and  getting  the 
unique  requirements  of  the  government  in  coherence  with  industry 
needs. 

Let  me  submit  for  the  record  my  testimony,  and  thank  you. 

[The  prepared  statement  of  Dr.  Wakid  follows:] 
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Chairwoman  Morella  and  Members  of  the  Subcommittee; 


This  is  a  time  of  challenge  and  a  time  of  change  for  the  National  Institute  of  Standards 
and  Technology  in  the  area  of  information  technology.  The  challenge  is  to  build  a  new 
organization  that  will  strengthen  NIST's  ability  to  provide  measurements  and  related 
standards  for  our  nation's  information  technology  industries.  To  meet  this  challenge,  we 
plan  to  establish  an  Information  Technology  Laboratory  that  will  be  built  on  the 
foundation  of  the  existing  Computer  Systems  Laboratory  (CSL)  and  the  Computing  and 
Applied  Mathematics  Laboratory  (CAML).  I  am  honored  to  speak  to  you  today  about 
how  CSL  and  CAML  are  working  with  industr\'  to  address  critical  problems  in 
information  technology  which  demand  the  generic  solutions  that  we  are  uniquely 
qualified  to  provide. 

The  NIST  laboratories  have  a  limited  but  critical  role  in  helping  to  meet  U.S.  industry's 
needs  for  measurement  and  standards.  The  Computer  Systems  Laboratory  and  the 
Computing  and  Applied  Mathematics  Laboratory  provide  the  technical  underpinnings 
and  enabling  tools  that  accelerate  getting  products  into  the  marketplace  and  help  everyone 
do  their  job  better. 

Information  technology  is  an  enabling  technology  that  is  being  embedded  in  consumer 
products,  household  appliances,  desktop  computers,  and  network  services.  It  is  becoming 
an  essential  tool  for  doing  business,  and  is  changing  the  way  that  we  do  business. 

We  direct  our  work  to  overcome  the  barriers  to  the  development  and  use  of  information 
technology  systems  that  are  secure,  interoperable,  easily  usable,  and  scalable.  Inadequate 
security  is  a  serious  barrier  to  the  use  of  open  networks  for  exchanging  sensitive 
information,  or  for  carrying  out  high  value  transactions.  Systems  that  lack 
interoperability  and  are  difficult  to  use  make  it  frustrating  for  users  who  would  like  to 
change  their  traditional  ways  of  doing  business  but  are  reluctant  to  adopt  new  technology. 
Even  successful  applications  often  cannot  be  used  on  a  mass  basis  because  they  cannot 
scale  up  to  larger  proportions. 

We  are  developing  accurate  and  reliable  tests,  test  methods,  and  reference  data,  which  are 
needed  by  many  producer  and  user  organizations  to  overcome  these  barriers.  Our  work  is 
often  done  in  collaboration  with  industry,  and  always  done  with  objectivity  and 
neutrality.  Our  activities  cover  a  broad  range  of  networking,  security,  and  advanced 
information  technologies;  mathematical,  statistical  and  computational  sciences;  and  the 
operation  and  modernization  of  computing  and  telecommunications  facilities  within 
NIST. 


Ongoing  Responsibilities 

Like  our  counterpart  NIST  laboratories,  our  main  emphasis  is  on  the  development  of 
tests,  test  methods  and  reference  data.  But  we  also  continue  to  carry  out  some  basic 
functions.  CSL  continues  to  be  the  government's  source  of  technology  and  expertise  for 
technical  information  technology  standards  including  those  needed  to  protect  sensitive, 
but  unclassified  information.  CAML  continues  to  provide  computational, 
supercomputing,  networking  and  administrative  computing  services  to  all  of  NIST.  The 
laboratory  develops  and  applies  mathematical  and  statistical  algorithms  and  models  in 
collaboration  with  NIST  scientists  and  engineers,  and  their  industrial  collaborators.  These 
are  important,  ongoing  responsibilities. 

Three  Focus  Areas 

Today,  I  would  like  to  discuss  three  areas.  The  first  area  is  test  methods  for 
communications,  human-machine  interfaces,  computer  security,  and  software.  For 
security  and  software  especially,  these  activities  represent  a  shift  from  past  efforts.  We 
are  moving  from  the  development  of  standards  for  maturing  technologies  to  the 
development  of  open,  precompetitive  tests  and  test  methods.  Although  both  needs  are 
important,  we  have  come  to  realize  that  reliable  test  methods  help  to  stimulate  the  growth 
and  adoption  of  new  technology.  This  is  where  our  programs  can  make  a  real  difference. 

The  second  focus  area  is  algorithms  and  analytic  tools  for  mathematics  and  statistics  to 
improve  understanding  of  complex  processes  and  analysis  of  data.  The  third  focus  area  is 
our  responsibility  to  federal  agencies  under  the  Information  Technology  Management 
Improvement  Act  of  1996  and  Office  of  Management  and  Budget  Circular  A- 130. 

Test  Methods 

Our  work  on  test  methods,  tests,  standard  reference  data,  and  tests  for  conformance  to 
written  standards  helps  producers  develop  better,  higher  quality  products  and  helps  users 
evaluate  these  products.  Information  technology  systems  are  so  complex  that  it  is 
virtually  impossible  to  evaluate  correctness  or  performance  of  the  various  parts  without 
automated  tests  and  methods.  These  tests  and  methods  are  most  useful  and  cost  effective 
when  they  are  widely  adopted,  but  they  are  beyond  the  reach  of  individual  companies  or 
user  organizations  to  develop. 

As  a  neutral  party,  we  are  often  asked  to  help  industry  standards  writing  organizations  by 
developing  tests  that  capture  the  technical  description  of  the  written  standards.  The  tests 
help  to  assure  compliance  and  consistency  of  products  that  implement  the  standards.  We 
do  not  set  the  standards.  Industry  sets  standards.  We  focus  on  advanced  technologies 
that  are  not  widely  implemented  or  not  available  in  commercial  off-the-shelf  products. 


Here  are  some  examples  of  how  our  work  is  used  by  a  broad  spectrum  of  organizations  to 
advance  the  development  and  use  of  new  technology.  We  chose  to  participate  in  these 
projects  because  they  fit  into  our  unique  mission,  have  a  high  probability  of  success,  and 
will  have  a  high  potential  impact  if  successful. 

•  We  have  been  collaborating  with  industry's  Digital  Audio/Video 
Interoperability  Council  (DAVIC)  to  develop  tests  for  the  interoperability 
of  different  video  applications  for  the  home.  Interoperability  is  critical  to 
the  delivery  of  digital  video  information  through  a  variety  of  different 
networks  that  have  different  requirements  for  handling  data. 

•  We  have  been  cooperating  with  the  ATM  Forum,  an  industry  consortium 
developing  standards  for  Asynchronous  Transfer  Mode  (ATM) 
technology.  We  are  contributing  to  the  development  of  tests  for  the 
interoperability  of  products  and  services  based  on  this  new,  high  speed 
digital  technology  for  networks.  ATM  will  deliver  a  complex  mix  of 
digital  information  including  text,  images,  video,  and  sound  to  the  desktop 
if  consistent  standards  can  be  developed  and  used  properly. 

•  We  were  invited  by  a  standards  committee  of  the  Institute  of  Electrical  and 
Electronic  Engineers  (IEEE  802.14)  to  help  the  committee  evaluate 
proposals  for  protocol  standards  for  coaxial  cable  modems.  This 
equipment  is  expected  to  bring  a  new  range  of  interactive  TV  and  other 
interactive  computing  services  to  consumers. 

•  We  developed  a  method  for  evaluating  the  performance  of  optical 
character  recognition  (OCR)  systems  that  process  handprinted  text.  These 
systems  automate  the  conversion  of  handprinted  letters  and  numbers  to  a 
digital  form  for  computer  processing.  Automatic  recognition  of  different 
styles  of  handprinting  continues  to  be  a  tough  technical  challenge.  We 
have  distributed  our  evaluation  method  and  a  collection  of  many  different 
samples  of  hand  printed  text  to  more  than  150  U.S.  companies,  small  and 
large.  Many  of  these  organizations  are  using  the  system  to  test  their  own 
products  or  to  develop  new  products  that  perform  better  and  faster. 

•  We  have  produced  tests  for  evaluating  the  performance  of  speech 
recognition  systems.  Speech  recognition  technology  has  long  been 
thought  to  be  a  good  way  for  humans  to  interact  with  computers  by 
speaking  to  them.  But  these  systems  must  still  be  improved  to  work  no 
matter  who  the  speaker  is  and  to  recognize  words  spoken  by  humans  using 
their  natural  speech  patterns.  Major  companies  and  universities  involved 
in  speech  recognition  research  have  relied  on  our  tests  to  evaluate  and 
improve  their  systems.  Our  sponsors  of  this  work  cite  the  tests  as 


influential  drivers  of  continuing  improvements  that  are  occurring  in 
automated  speech  recognition  systems.  The  mari^et  potential  is  enormous. 

•  We  are  developing  measures  of  performance  for  a  prototype  system  that 
will  enable  humans  to  get  information  from  a  large  computerized  library 
collection  of  data.  This  system  will  demonstrate  how  people  could  ask 
questions  to  the  computer  and  receive  responses  about  the  contents  of  the 
documents  in  the  library.  We  expect  that  the  measures  of  performance 
will  help  to  advance  the  commercial  availability  of  such  systems. 

•  Our  work  with  the  North  American  Integrated  Services  Digital  Network 
(ISDN)  Users  Forum  to  develop  standards  and  conformance  tests  for 
ISDN  technology  made  significant  contributions  toward  the  development 
of  the  technology.  This  was  the  conclusion  of  a  study  done  on  the  Forum's 
impact  on  U.S.  industry.  The  study,  based  on  industry  estimates,  projected 
the  economic  rate  of  return  from  our  investment  in  ISDN  to  be  156 
percent. 

Mathematical,  Computational  and  Statistical  Sciences 

Next,  I  would  like  to  turn  to  our  activities  in  mathematical,  computational,  and  statistical 
sciences.  A  recent  report  on  the  needs  of  the  chemical  industry  for  new  technology 
specifically  identified  improvements  in  modeling  and  the  application  of  information 
technology  as  essential  to  the  chemical  process  industry.  These  requirements  permeate 
many  other  industry  sectors  as  well. 

NIST  mathematicians  model  physical  phenomena  to  enable  our  scientists  to  visualize  and 
understand  them.  The  methodology  and  algorithms  developed  to  solve  NIST's  owti 
research  problems  are  also  shared  with  industry  and  academia.  NIST  statisticians 
develop  and  apply  new  experiment  designs  to  help  scientists  and  engineers  work 
productively,  model  their  data,  and  assess  its  uncertainty  so  that  their  collaborators  can 
draw  correct  conclusions  from  their  experiments.  For  example,  NIST  statisticians, 
mathematicians,  and  engineers  are  developing  a  new  calibration  model  for  Coordinate 
Measuring  Machines  that  can  be  used  to  improve  the  determination  of  part  dimensions  in 
manufacturing.  Since  over  20,000  Coordinate  Measuring  Machines  are  in  use  throughout 
the  country,  better  modeling  provided  substantial  savings  to  U.S.  industry  through 
improved  quality  and  productivity  and  significant  reductions  in  waste. 

NIST's  Guide  to  Available  Mathematical  Software  (GAMS)  has  been  a  very  successful 
effort  to  disseminate  information  about  mathematical  software  that  can  be  used  in 
scientific  and  engineering  applications.  GAMS  helps  scientists  and  engineers  locate  and 
use  nearly  10,000  reusable  problem-solving  modules  from  currently  available 
mathematical  software  packages.  Since  we  made  it  available  on  a  World  Wide  Web 
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server  in  April  1994.  GAMS  has  received  more  than  1.800,000  inquiries,  and  has  been 
awarded  the  highest.  4-star  rating  by  the  McKinley  Group,  a  private  organization  that 
rates  Web  services. 

We  have  developed  a  program  that  will  extend  our  expertise  in  modeling  and  simulation 
to  more  directly  assist  scientists  and  engineers  in  industry.  For  example,  we  plan  to 
develop  reference  data  sets  that  will  help  software  developers  improve  the  reliability  of 
their  mathematical  and  statistical  software  products.  Currently,  there  are  very  few  such 
tools  available.  The  algorithms  and  the  software  are  difficult  to  test  objectively;  errors 
occur  often  in  specialized  circumstances  that  are  related  to  the  problem  to  be  solved.  We 
plan  to  identify  and  make  available  neutral  and  objective  test  data  to  assist  developers, 
researchers,  and  users  in  making  these  difficult  evaluations. 

Federal  responsibilities 

Last,  I  would  like  to  reaffirm  our  commitment  to  serving  other  federal  agencies.  The 
research  in  test  methods  and  measurements  that  I  discussed  earlier  will  effectively  serve 
the  federal  government.  These  test  methods  will  drive  the  development  of  higher  quality 
and  better  hardware  and  software  products.  This  will  have  a  significant  payoff  for  federal 
buyers  of  these  products,  and  will  also  benefit  private  sector  users. 

I  would  like  to  conclude  with  a  brief  discussion  of  our  responsibilities  for  computer 
security,  and  specifically  for  cryptography.  You  are  probably  well  aware  of  the  risks  to 
our  nation's  and  our  government's  computer  resources,  and  the  challenges  in  securing 
those  resources.    NIST  plays  a  key  role,  under  the  Computer  Security  Act  of  1987,  in 
developing  standards  and  guidelines  to  assist  federal  departments  and  agencies  in 
securing  their  automated  information  assets. 

Adequately  and  appropriately  securing  automated  information  resources  is  not  a  simple 
or  one-dimensional  task.  There  is  no  silver  bullet,  such  as  cryptography,  that  renders  a 
system  and  the  data  it  processes  secure.  Adequately  protecting  a  system  requires  that  the 
risks  be  studied,  and  appropriate  controls  be  selected  and  implemented  correctly.  These 
include  management,  technical  and  operational  controls,  encryption,  authentication,  and 
incident  handling.  Implemented  correctly,  such  controls  operate  synergistically  to 
provide  security. 

Our  computer  security  program  is  currently  focused  on  five  key  areas  where  we  believe 
we  can  make  the  greatest  contribution  to  solving  the  overall  problem; 

•  advanced  authentication  technology, 

•  security  criteria  and  evaluation. 

•  secure  Internet  and  network  connectivity, 

•  cr>'ptographic  technologies,  and 
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•  public  key  infrastructure  activities. 

Our  efforts  in  cr>'ptography  and  particularly  encryption  receive  the  most  public  attention. 
Although  encryption  can  be  used  as  a  cost-effective  tool  to  protect  data  against 
unauthorized  disclosure,  it  can  also  be  used  to  thwart  the  legitimate  missions  of  law 
enforcement  and  national  security  agencies.  We  have  become  involved  in  this  issue 
because  of  our  legislative  mandate  to  provide  agencies  with  sound  security  advice.  At  the 
same  time,  we  must  meet  our  obligations  not  to  take  actions  that  may  harm  society. 

We  work  with  industry,  law  enforcement,  and  security  officials  to  help  them  in  their 
efforts  to  develop  sound  cryptographic  techniques  that  balance  these  interests.  Many  of 
the  techniques  that  we  have  developed  have  been  voluntarily  adopted  by  private  sector 
organizations.  An  example  is  the  Data  Encryption  Standard,  which  is  based  on  a 
commercially  developed  technique  and  which  long  ago  became  a  voluntary  industry 
standard  for  protecting  computer  information. 

As  a  leader  in  computer  security  for  the  federal  government,  we  are  often  asked  by 
governmental  organizations  of  other  countries  to  participate  in  joint  projects.  We  are 
currently  cooperating  with  Canada.  France,  the  UK.  Germany,  and  the  Netherlands  to 
develop  common  evaluation  criteria  for  computer  security  products  and  systems.  This 
cooperative  approach  is  expected  to  pay  off  for  all  participants.  One  uniform, 
internationally  accepted  method  for  evaluating  computer  security  products  would  lower 
costs  for  producers,  increase  the  availability  of  products  for  users,  and  help  to  broaden  the 
global  market  for  U.S.  products. 

Summary 

We  are  committed  to  expand  and  strengthen  our  collaboration  with  industry  to  carr%  out 
our  programs.  We  are  committed  to  advancing  the  development  of  pivotal,  forward- 
looking  technologies  by  removing  impediments  caused  by  the  lack  of  appropriate 
measurement  technology  or  by  inappropriate  standards.  The  timing  is  right. 
Interoperability  and  security  of  information  technology  products  and  services  are 
essential  to  a  global  marketplace  for  U.S.  industry.  Better  use  of  new  mathematical  and 
statistical  algorithms  and  models  for  data  analysis  and  decision  making  will  help  the 
United  States  move  forward  aggressively. 

Information  technology  is  a  big  $630  billion  industry  in  the  U.S.  We  cannot  ignore  that 
large  part  of  the  economy  and  the  opportunity  that  it  provides  for  our  industry. 

We  believe  that  we  can  be  successful  in  accomplishing  our  goals,  and  that  our  work  at 
NIST  can  make  a  difference.  Our  work  is  generic.  It  provides  the  infrastructure  that 
individual  organizations  do  not  invest  in  by  themselves,  but  which  enable  all  to  move 
forward  in  achieving  secure,  interoperable,  usable,  and  scalable  systems.  If  we  are 
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successful  in  these  efforts,  our  industry  will  be  well  positioned  to  meet  future  challenges 
and  changes. 
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the  Presidential  Rank  Award  of  Meritorious  Executive  in  1993  and  the  Department  of 
Commerce's  Silver  Medal  in  1992. 
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Mrs.  Morella.  Thank  you,  Dr.  Wakid.  Your  entire  testimony  that 
you  submitted  will  be  included  in  the  record  in  its  totality. 
Dr.  Roskies? 

STATEMENT  OF  DR.  RALPH  Z.  ROSKIES 

Dr.  Roskies.  Chairwoman  Morella  and  Members  of  the  Sub- 
committee, I  am  pleased  and  privileged  to  address  you  today  as  Co- 
Chair  of  the  National  Research  Council's  Panel  for  Information 
Technology,  which  conducts  an  annual  assessment  of  the  programs 
of  the  Computing  and  Applied  Mathematics  Laboratory  and  the 
Computer  Systems  Laboratory  at  the  National  Institute  of  Stand- 
ards and  Technology. 

My  own  training  is  in  theoretical  high  energy  physics.  I  am  a 
professor  of  physics  at  the  University  of  Pittsburgh  and  I  am  Sci- 
entific Director  of  the  Pittsburgh  Supercomputing  Center.  I  have 
been  involved  in  the  National  Research  Council  assessments  of  the 
NIST  laboratories  since  1991.  The  Panel  for  Information  Tech- 
nology currently  has  24  members,  of  whom  seven  work  in  aca- 
demia,  12  in  industry,  and  five  work  in  other  sectors,  such  as  gov- 
ernment or  private  non-profits. 

Historically,  NIST  has  played  an  important  role  in  the  develop- 
ment of  information  technology  by  identifying  and  developing  key 
enabling  technologies  and  technology  components,  by  establishing 
standards  and  protocols,  and  by  establishing  technical  specifica- 
tions for  Federal  procurement. 

More  recently,  the  unprecedented  pace  of  development  and  rapid 
commercialization  of  information  technology  has  meant  that  NIST 
must  redefine  its  role  in  order  to  respond  more  effectively  to  the 
needs  of  industry.  Rather  than  defining  standards,  NIST  should 
serve  as  a  facilitator  for  standards  development  and  as  a  conform- 
ance tester,  emphasizing  interoperability,  reliability,  and  usability. 

Accordingly,  the  two  laboratories  at  NIST  that  have  traditionally 
focused  on  computer  technology,  computer  science,  and  applied 
mathematics  are  being  merged  into  one  organization  unit  known  as 
the  Information  Technology  Laboratory.  The  new  unit  will  have 
both  the  important  task  of  supporting  advanced  computational  and 
analytical  needs  of  the  other  science  and  engineering  units  within 
NIST  and  facilitating  the  growth  of  information  technology  through 
research  and  development  in  advanced  networking  technologies, 
visualization,  human-machine  interfaces,  computer  and  network  se- 
curities, and  standards  and  technology  to  support  electronic  com- 
merce. 

The  panel  supports  the  decision  of  NIST  management  to  merge 
the  Computing  and  Applied  Mathematics  and  Computer  Systems 
Laboratories  into  a  new  Information  Technology  Laboratory  that 
will  be  more  tightly  focused  on  the  needs  of  industry.  While  the 
final  shape  of  an  Information  Technology  Laboratory  will  be  influ- 
enced by  many  factors  which  we  cannot  yet  foresee,  we  believe  that 
this  reorganization  will  help  NIST  respond  more  effectively  to  the 
nation's  needs  for  standards  and  technology  in  the  information 
technology  arena. 

Thank  you  for  the  opportunity  to  appear  before  you  today. 

[The  prepared  statement  and  attachment  of  Dr.  Roskies  follow:] 
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Ms  Chairwoman  and  Members  of  the  Subcommittee: 

I  am  pleased  and  privileged  to  address  you  today  as  co-Chair  of  the  National  Research 
Council's  (NRC)  Panel  for  Information  Technology  which  conducts  an  annual  assessment  of  the 
programs  of  the  Computing  and  Applied  Mathematics  Laboratory  and  the  Computer  Systems 
Laboratory  at  the  National  Institute  of  Standards  and  Technology  (NIST)    My  own  training  is  in 
theoretical  high  energy  physics,  and  I  am  a  Professor  of  Physics  at  the  University  of  Pittsburgh 
and  Scientific  Director  of  the  Pittsburgh  Supercomputing  Center    I  have  been  involved  in  the 
NRC's  assessment  of  the  NIST  Laboratories  since  1991    The  Panel  for  Information  Technology 
currently  has  24  members,  of  whom  7  work  in  academe,  12  work  in  industry,  and  5  work  in  other 
sectors,  such  as  government  or  private  non-profits. 

I  would  like  to  read  into  the  record  the  accompanying  1995  Report  of  the  Panel  for 
Information  Technology  which  evaluates  in  detail  issues  of  balance,  effectiveness,  technical 
quality,  impact  on  industry  and  management  of  each  of  the  divisions  of  these  two  laboratories 

NIST  has  been  engaged  in  the  development  of  standards  for  and  key  components  of 
computer  and  information  technology  since  this  technology  and  its  precursors  first  emerged 
Much  of  the  immediate  post-World  War  II  progress  in  numerical  analysis  and  special  functions 
came  out  of  the  National  Bureau  of  Standards,  now  NIST    Likewise,  NIST  has  played  an 
important  role  in  developing  standards,  measures,  and  testbeds  such  as  digital  signatures  and  data 
interchange  formats  that  make  a  national  information  infrastructure  possible 

Historically,  NIST  has  played  an  important  role  in  the  development  of  information 
technology  by  identifying  and  developing  key  enabling  technologies  and  technology  components, 
by  establishing  standards  and  protocols,  and  by  establishing  technical  specifications  for  federal 
procurement    More  recently,  the  unprecedented  pace  of  development  and  rapid 
commercialization  of  information  technology  has  meant  that  NIST  must  redefine  its  role  in  order 
to  respond  more  effectively  to  the  needs  of  industry.  Rather  than  defining  standards,  NIST  should 
serve  as  a  facilitator  for  standards  development,  and  as  a  conformance  tester,  emphasizing 
interoperability,  reliability  and  usability  Accordingly,  the  two  laboratories  at  NIST  that  have 
traditionally  focused  on  computer  technology,  computer  science,  and  applied  mathematics  are 
being  merged  into  one  organizational  unit  known  as  the  Information  Technology  Laboratory. 

The  new  unit  will  have  both  the  important  task  of  supporting  the  advanced  computational 
and  analytical  needs  of  other  science  and  engineering  units  within  NIST,  and  facilitating  the 
growth  of  the  information  technology  through  research  and  development  in  advanced  networking 
technologies,  visualization,  human-machine  interfaces,  computer  and  network  security,  and 
standards  and  technology  to  support  electronic  commerce 

The  Panel  supports  the  decision  of  NIST  management  to  merge  the  Computing  and 
Applied  Mathematics  and  Computer  Systems  Laboratories  into  a  new  Information  Technology 
Laboratory  that  will  be  more  tightly  focused  on  the  needs  of  industry.  In  fact,  the  Panel's  chief 
recommendation  in  its  1995  assessment  was  that  the  merger  proceed  more  quickly    Since  the 
time  of  the  Panel's  site  visit  to  NIST  in  April  1995,  Shukri  Wakid  has  been  appointed  Acting 
Director  for  both  of  the  two  units  which  will  eventually  form  the  Information  Technology 
Laboratory,  and  planning  for  the  joint  laboratory  has  accelerated. 
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While  the  final  shape  of  an  Information  Technology  Laboratory  will  be  influenced  by  many 
factors  which  we  cannot  yet  foresee,  I  believe  that  this  reorganization  will  help  NIST  respond 
more  effectively  to  the  nation's  needs  for  standards  and  technology  in  the  information  technology 
arena. 


Thank  you  for  the  opportunity  to  appear  before  you  today. 
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Fiscal  Year  1995  Assessment  of  the 

Information  Technology  Laboratory 

by  the 

Panel  for  Information  Technology 


Panel  Members 

Ralph  Z.  Roskies,  University  of  Pittsburgh,  Co-Chair 

Michael  B.  Spring,  University  of  Pittsburgh,  Co-Chair 

Cheryl  Y.  Begandy,  Alcoa  Technical  Center 

Herbert  D.  Benington,  Consultant  to  UNISYS 

Mary  Ellen  Bock,  Purdue  University 

Allen  L.  Brown,  Jr.,  Xerox  Corporation 

Dona  Crawford,  Sandia  National  Laboratories 

Dorothy  E.  Denning,  Georgetown  University 

Pamela  G  Doctor,  Battelle  Pacific  Northwest  Laboratories 

Edward  A  Feigenbaum,  Chief  Scientist  of  the  Air  Force 

Avner  Friedman,  University  of  Minnesota 

James  George,  Los  Alamos  National  Laboratory 

Andrew  S  Grimshaw,  University  of  Virginia 

Carl  M.  Harris,  George  Mason  University 

Thomas  P.  Kehler,  CONNECT,  Inc. 

Ellen  M.  Knapp,  Coopers  &  Lybrand 

Sandra  M.  Lambert,  Citicorp 

Martin  C.  Libicki,  Institute  for  National  Strategic  Studies 

Carl  Machover,  Machover  Associates  Corporation 

Samuel  P.  Marin,  General  Motors  Research  and  Development  Center 

Jill  P.  Mesirov,  Consultant 

Stewart  D.  Personick,  Bell  Communications  Research 

Jerome  Sacks,  National  Institute  of  Statistical  Sciences 

David  F  Shanno,  Rutgers  University 

Daniel  L.  Solomon,  North  Carolina  State  University 

Ivar  Stakgold,  University  of  Delaware 

Raymond  T.  Yeh,  International  Software  Systems,  Inc. 

Eddie  L.  Zeitler,  Fidelity  Investments 

Submitted  for  the  panel  by  its  Co-Chairs,  Ralph  Z.  Roskies  and  Michael  B.  Spring,  this 
assessment  of  the  fiscal  year  1995  activities  of  the  Information  Technology  Laboratory  is  based  on 
site  visit  by  the  panel  on  May  1-3,  1995,  and  on  the  annual  report  of  the  laboratory. 
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LABORATORY  OVERVIEW 

Introduction 

The  Computer  Systems  Laboratory  (CSL)  and  the  Computer  and  Applied  Mathematics 
Laboratory  (CAML)  are  due  to  be  merged  into  a  single  entity  that  will  be  known  as  the 
Information  Technology  Laboratory  (ITL).  As  of  the  time  of  this  assessment,  the  necessary 
approvals  from  the  Congress  and  the  Department  of  Commerce  for  this  planned  merger  have  not 
been  gained  nor  has  a  director  for  the  new  laboratory  been  identified 

What  follows  is  an  assessment  of  the  divisions  of  the  two  constituent  laboratories,  the 
Computer  Systems  Laboratory  and  the  Computer  and  Applied  Mathematics  Laboratory.  In  light 
of  the  unique  nature  of  the  assessment  this  year,  the  overall  laboratory  assessment  focuses  on  the 
planning  for  the  new  Information  Technology  Laboratory,  and  the  technical  evaluations  are  given 
in  the  divisional  assessments. 


Planning  for  the  Information  Technology  Laboratory 

The  panel  recognizes  the  difficult  task  faced  by  the  divisions  of  the  two  laboratories  in 
carrying  out  their  mission  during  a  period  of  planning  for  a  broader  and  refocused  mission  as  part 
of  the  future  Information  Technology  Laboratory.  This  also  occurs  at  a  time  in  which  most  of  the 
divisions  must  prepare  to  move  their  operations  to  a  new  off"-campus  location  designated  as  NIST 
North.  At  the  same  time,  the  panel  believes  that  the  planning  eff"ort  for  the  new  Information 
Technology  Laboratory  must  move  forward  more  rapidly    Information  technology  is  a  fast 
moving  field,  and  NIST  must  become  more  agile  and  more  positively  engaged  if  it  is  to  play  a 
significant  role  in  this  area.  Although  the  laboratories  and  the  divisions  have  worked  diligently  to 
move  forward  with  the  planning  process,  they  are  now  at  a  point  where  flirther  progress  requires 
dedicated  leadership  at  the  laboratory  level.  It  is  important  that  a  director  for  the  Information 
Technology  Laboratory  be  appointed  immediately,  on  an  interim  basis  if  necessary,  in  order  to 
lead  the  process  of  shaping  the  new  laboratory.  Although  thecurrent  directors  of  the  Computer 
Systems  Laboratory  and  the  Computer  and  Applied  Mathematics  Laboratory  have  cooperated  in 
the  development  of  the  Information  Technology  Laboratory,  it  has  been  difficult  for  them  to 
identify  opportunities  for  the  new  laboratory  and  infuse  the  staff  with  enthusiasm  at  the  prospect 
of  playing  a  significant  role  in  the  development  of  a  national  information  technology 
infrastructure. 

Although  there  is  considerable  dynamism  at  the  division  and  group  level,  uncertainty  about 
the  future  course  of  the  Information  Technology  Laboratory  and  the  slow  pace  of  the  planning 
process  have  had  a  negative  impact  on  morale.  The  lack  of  direction  resulting  from  NIST's 
failure  to  identify  a  director  for  the  new  laboratory  has  placed  a  serious  constraint  on  the  priority- 
setting  process  at  the  laboratory  level  and  has  also  negatively  affected  the  planning  process  at  the 
division  level.  The  goals  of  the  new  laboratory  must  be  articulated  well  enough  for  the  divisions 
to  incorporate  them  into  their  own  priorities. 

In  several  divisions,  historical  program  interests  still  dominate  the  priority-setting  process 
and  divisional  management  needs  to  be  more  proactive  in  setting  program  directions    At  the 
laboratory  level,  a  planning  document  has  been  developed  that  identifies  six  areas  of  concentration 
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for  the  Information  Technology  Laboratory:  interoperability,  security,  services,  mathematics 
(including  modeling,  statistics,  and  associated  software),  human-machine  interfaces,  and 
applications  development.  Interoperability  and  human-machine  interfaces  are  both  critical  thrusts 
for  the  new  laboratory.   Security  is  a  natural  fit  to  the  mission  of  the  new  laboratory  and  to  the 
expertise  currently  available  in  the  Computer  Security  Division.  The  applications  development 
thrust  is  the  least  well  developed  of  the  six  areas  of  concentration. 

The  laboratory's  planning  for  the  mathematics  thrust,  which  includes  statistical  and 
computational  sciences,  must  take  account  of  the  special  mission  of  the  Statistical  Engineering 
Division  and  the  Applied  and  Computational  Mathematics  Division  to  provide  mathematical, 
statistical,  and  computational  support  to  other  NIST  laboratories. 

With  respect  to  industrial  impact,  the  potential  of  the  new  laboratory  is  cleariy  substantial 
The  existing  divisions  can  identify  anecdotal  evidence  of  positive  industrial  impact,  but  this  often 
appears  to  be  a  consequence  of  achieving  other  goals  rather  than  the  result  of  an  explicit  strategy 
to  improve  the  competitiveness  and  productivity  of  the  US  information  technology  industry. 
With  the  creation  of  the  new  laboratory  and  its  stated  goal  of  enhancing  the  competitive  position 
of  US  industry,  more  deliberate  planning  for,  and  evaluation  of,  industrial  impact  is  required. 


Recommendations 

The  following  are  the  panel's  recommendations  for  the  Information  Technology 
Laboratory  as  a  whole. 

The  director  of  NIST  should  set  a  more  specific  charge  for  the  Information  Technology 
Laboratory  and  move  with  all  due  speed  to  appoint  a  laboratory  director. 

All  levels  of  management,  from  the  director  of  NIST  down  through  division  chiefs,  should 
communicate  more  effectively  to  the  staff  the  goals  of  the  new  laboratory  and  the  opportunities 
they  represent. 

The  panel  observes  that  statistical  support  of  metrology,  calibration,  and  Standard  Reference 
Materials  (SRMs)  and  applied  mathematical  and  statistical  support  of  research  efforts  in  other 
NIST  laboratories  are  essential  to  the  mission  of  NIST  and  recommends  that  these  services  should 
be  continued  regardless  of  the  administrative  structure  of  the  Information  Technology  Laboratory. 

The  panel  recommends  that  laboratory  and  division  management  take  a  proactive  approach  to 
selecting  current  laboratory  activities  and  projects  so  as  to  optimally  position  the  new  Information 
Technology  Laboratory  based  on  an  assessment  of  where  information  technology  will  be  going  in 
the  next  5  years. 

Note;  Subsequent  to  the  site  visit  by  the  1995  Assessment  Panel,  the  chief  of  the  Advanced 
Systems  Division  within  the  Computer  Systems  Laboratory  was  appointed  acting  director  of  the 
Information  Technology  Laboratory. 
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Laboratory  Responses  to  Fiscal  Year  1994  Recommendations 

The  following  is  the  laboratory's  response  to  the  recommendations  of  the  1994 
Assessment  Panel.  The  1995  panel  commented  on  those  responses  where  appropriate,  and  those 
comments  are  included  here.  In  fiscal  year  1994,  the  Computing  and  Applied  Mathematics 
Laboratory  and  the  Computer  Systems  Laboratory  were  assessed  individually.  Recommendations 
and  responses  are  arranged  accordingly. 


Computing  and  Applied  Mathematics  Laboratory 

1994  Panel  Recommendation:  ".  .  .[The]  panel  recommends  strongly  that  division  chiefs  be 
intimately  involved  in  developing  the  mission  statement  and  organizational  structure  of  the 
merged  [Information  Technology]  laboratory"  (p  233). 

Laboratory  Response:  The  planning  committee  for  the  Information  Technology  Laboratory 
includes  all  of  the  division  chiefs. 

1994  Panel  Recommendation:  "The  panel  urges  that  mathematical  and  statistical  programs  be 
continued  under  ITL  .  .  ."  (p  233) 

Laboratory  Response:  The  planning  committee  has  identified  mathematics,  statistics,  and 
scientific  computing  as  a  strategic  program  area  for  the  Information  Technology  Laboratory. 

1994  Panel  Recommendation:  "The  panel  urges  C  AML  to  add  new  vigor  to  its  expertise  by 
hiring  new  staff  and  by  expanding  its  visitor  and  postdoctoral  fellowship  program"  (p  233) 
Laboratory  Response:  The  Computing  and  Applied  Mathematics  Laboratory  will  continue  to 
encourage  visitors  and  to  seek  postdoctoral  research  associates 

1995  Panel  Comment:  The  Computing  and  Applied  Mathematics  Laboratory  has  indeed 
encouraged  visitors  and  has  explored  the  possibility  of  contracting  out  some  services. 


Computer  Systems  Laboratory 

1994  Panel  Recommendation:  "CSL  management  must  careRilly  approach  the  merger  with 
CAML  to  form  ITL  in  order  to  maintain  staff  morale  and  support  CSL  core  capabilities.  The  staff 
of  the  two  laboratories  should  work  together  to  define  the  challenges  and  opportunities  of  the 
merger,  providing  input  on  the  organization  and  programs  to  the  NIST  director"  (p.  202). 
Laboratory  Response:  An  ITL  Planning  Group  has  been  formed  and  has  held  regular  meetings 
since  September  1994  to  review  the  existing  programs  and  to  plan  the  steps  necessary  to  create 
the  Information  Technology  Laboratory.  CSL  and  CAML  managers  meet  regularly  in  joint 
management  meetings. 

1995  Panel  Comment:  Although  CSL  and  CAML  managers  have  worked  together  on  the  ITL 
planning  document,  there  is  no  evidence  that  they  have  explored  the  opportunities  for  synergy  that 
this  merger  could  create. 
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1994  Panel  Recommendation:  "CSL  must  work  creatively  to  overcome  its  severe  staffing 
limitations.  Inclusion  of  part-time  faculty  and  students  in  the  FTE  [full-time  equivalent]  ceiling  is 
counterproductive;  guest  staffing  should  be  controlled  by  funding,  space,  and  mutual  benefit"  (p. 
202). 

Laboratory  Response:  CSL  continues  to  use  part-time  faculty  and  students  where  appropriate  to 
carry  out  program  responsibilities.  Guest  staff  members  are  encouraged  when  space  is  available 
and  there  are  programmatic  benefits.  Currently,  staffing  is  limited  by  funding  constraints  and  by 
FTE  ceilings. 

1994  Panel  Recommendation:  "CSL  needs  larger,  more  flexible  space.  .  .  ,"  (p.  202). 
Laboratory  Response:  Plans  have  been  developed  for  a  leased  building  that  will  provide  space  for 
CSL  and  CAML  staff. 

1994  Panel  Recommendation:  "CSL  should  .  .  .  develop  a  model  for  its  customer  base  .  .  ." 
(p.  202). 

Laboratory  Response:  CSL  and  CAML  are  jointly  developing  a  strategic  planning  document  that 
covers  customer  and  technology  bases,  priorities,  and  methods  for  measuring  results 

1994  Panel  Recommendation:  "Each  CSL  division  should  upgrade  and  perfect  its  multiyear 
plans,  with  detailed  activity  descriptions,  milestones,  forecast  completion  dates,  funding  sources, 
staffing,  and  relations  to  other  NIST  projects.  CSL  divisions  should  plan  jointly.  Joint  programs 
involving  more  than  one  division  (or  laboratory)  should  have  a  single  plan,  showing  the  activities 
of  each  division,  group,  or  laboratory"  (p.  202) 

Laboratory  Response:  Joint  planning  among  CSL  divisions  has  been  ongoing  for  the  past  year. 
The  ITL  planning  document  identifies  six  thrust  areas  that  support  participation  across  the 
activities  of  several  divisions    Fiscal  year  1996  and  1997  initiatives  were  planned  on  a  joint  basis 
and  propose  work  that  will  be  carried  out  jointly  by  more  than  one  division. 

1995  Panel  Comment:  Although  CSL  divisions  plan  jointly,  existing  plans  vary  widely  in 
substantive  content,  completeness,  and  format    Some  divisions  have  no  formal  project  planning 
other  than  minimal  information  in  individual  performance  plans    It  would  be  helpful  if  a  template 
was  developed  and  distributed  to  all  divisions  so  that  project  planning  would  be  facilitated    The 
Computer  Security  Division  has  very  good  plans,  which  could  be  used  as  a  prototype  on  which  to 
build  the  template. 

1994  Panel  Recommendation:  "The  panel  recommends  a  skills  survey  and  a  summary  of 
available  versus  needed  skills,  which  could  be  used  to  guide  hiring  and  transfer  within  CSL" 
(p.  202) 

Laboratory  Response:  The  Computer  Systems  Laboratory  conducts  a  survey  of  skills  when  new 
programs  are  initiated  and  staff  members  are  assigned  to  new  efforts    An  overall  skills  survey 
would  be  more  beneficial  when  the  Information  Technology  Laboratory  is  formed  and  its 
programs  have  been  determined. 

1995  Panel  Comment:  Although  the  CSL  response  indicates  that  the  survey  would  be  more 
beneficial  when  ITL  programs  have  been  determined,  the  panel  believes  that,  in  fact,  the  skills 
survey  should  be  done  now,  since  it  will  be  helpful  in  the  reorganization  planning  and  in 
identifying  skill  sets  that  mav  be  required  in  the  new  laboratory. 
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1994  Panel  Recommendation:  "CSL  should  assess  the  value  of  its  'best-in-class'  examples  of 

projects  that  support  industrial  competitiveness.  These  'best-in-class'  projects  (with 

authentication  by  external  experts)  could  be  used  as  project  role  models  and  justification  for 

future  activities"  (p.  202). 

Laboratory  Response:  The  Computer  Systems  Laboratory  continues  to  evaluate  its  successful 

projects,  to  draw  on  the  lessons  learned,  and  to  apply  those  insights  to  the  design  of  future 

projects. 

1994  Panel  Recommendation:  "CSL's  formal  and  informal  networking  with  industry  appears  to 

be  increasing  and  should  continue  to  be  a  priority"  (p.  202). 

Laboratory  Response:  The  laboratory  is  pleased  that  its  increased  interactions  with  industry  have 

been  productive  and  plans  to  continue  to  work  with  industry  in  areas  where  it  can  make  a 

difference. 

1994  Panel  Recommendation:  "CSL  needs  to  adopt  clear  and  consistent  metrics  to  monitor  and 
assess  its  work.  For  example,  CSL  should  categorize  desktop  computers  used  in  the  emerging 
Nil  [national  information  infrastructure]  and  track  the  extent  to  which  CSL  standards,  guidelines, 
and  other  products  are  used  in  those  computers"  (p.  203). 

Laboratory  Response:  The  laboratory  agrees  on  the  need  to  monitor  and  assess  the  impact  of 
CSL  activities  and  will  include  metrics  for  measuring  the  impact  of  programs  in  the  strategic  plan 
that  is  being  developed. 


DIVISIONAL  ASSESSMENTS 

Applied  and  Computational  Mathematics  Division 

Mission 

The  Applied  and  Computational  Mathematics  Division  is  charged  with  providing  technical 
leadership  within  NIST  in  modem  analytical  and  computational  methods  for  solving  mathematical 
problems  of  interest  to  U.S.  industry.  This  work  includes  the  development  and  analysis  of 
theoretical  descriptions  of  phenomena  (mathematical  modeling),  the  design  and  analysis  of 
requisite  computational  methods  and  experiments,  the  transformation  of  these  methods  into 
efficient  numerical  algorithms  for  high-performance  computers,  the  implementation  of  these 
methods  in  high-quality  mathematical  software,  and  the  distribution  of  this  software  to  potential 
clients. 

Resources 

Funding  for  fiscal  year  1994  totaled  $3.7  million,  of  which  $2.8  million  was  from  NIST 
Scientific  and  Technical  Research  and  Services  (STRS),  $0.5  million  was  from  the  High- 
Perfonmance  Computing  and  Communications  initiative,  and  $0.4  million  was  from  other 
government  agencies  (OA).  The  division  employs  27  full-time  staff,  4  of  whom  are  in  Boulder. 
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The  Priority-Setting  Process 

Most  individual  projects  are  initiated  by  researchers  with  guidance  from  group  leaders. 
Many  of  these  are  joint  work  with  other  MIST  laboratories;  others  represent  the  development  of 
infrastructural  technologies  motivated  from  needs  observed  or  anticipated  after  consultations  with 
NIST  researchers.  Group  leaders  attempt  to  maintain  a  balance  of  projects  that  represent 
compatibility  with  divisional  program  thrusts,  relevance  to  NTST  programs,  significant  impact 
outside  of  NIST,  and  excellent  science. 

This  year,  extensive  additional  planning  has  centered  on  the  joint  efforts  of  the  Computing 
and  Applied  Mathematics  Laboratory  and  the  Computer  Systems  Laboratory  to  define  the 
motivation,  vision,  goals,  and  program  thrusts  for  the  Information  Technology  Laboratory 


Assessment  of  Priorities  and  the  Priority-Setting  Process 

The  division's  priorities  are  defined  in  terms  of  six  program  thrusts  developed  as  part  of  its 
1994  strategic  plan.  These  thrusts  are  appropriate  in  the  context  of  the  division's  stated  mission 
The  thrusts  encompass  current  areas  of  research  and  identify  important  new  areas  of  opportunity 
such  as  computational  metrology. 

The  priority-setting  process  for  project  and  program  selection  applies  compatibility  with 
the  current  programs  as  one  of  its  principal  selection  criteria    This  means  that  the  interests,  skills, 
and  preferences  of  division  staflf  together  with  program  history  will  continue  to  dominate  in 
setting  the  research  agenda  of  the  division    It  also  means  that  division  planning  tends  to  be  short 
term  as  it  is  generally  bound  by  the  time  horizon  of  the  collection  of  currently  active  projects 
Division  management  ought  to  engage  in  longer-range  planning 

In  view  of  the  changing  mandate  for  the  division  associated  with  the  formation  of  the 
Information  Technology  Laboratory,  division  management  needs  to  become  more  active  in  setting 
program  directions  that  will  result  in  appropriate  changes  in  project  content    In  particular,  there  is 
a  need  to  review  regularly  the  thrust  areas  to  identify  and  replace  lower-priority  thrust  areas  with 
new  areas  that  will  emphasize  mathematical  topics  that  address  or  enable  mainstream  research  in 
information  technology.  Project  impact,  as  measured  by  the  potential  customer,  should  play  a 
greater  role  in  the  priority-setting  process  within  the  division. 


Assessment  of  Technical  Programs 

There  is  a  long  and  notewonhy  tradition  of  research  in  applied  and  computational 
mathematics  at  NIST.  Indeed,  it  is  fair  to  say  that  much  of  the  immediate  post-Worid  War  II 
progress  in  numerical  analysis  and  special  fijnctions  came  out  of  the  National  Bureau  of 
Standards,  now  the  National  Institute  of  Standards  and  Technology.  Overall,  the  present  research 
program  in  the  Applied  and  Computational  Mathematics  Division  is  of  high  quality  and  deals 
principally  with  materials  modeling,  modeling  in  physics  and  chemistry,  computational  geometry 
and  metrology,  image  analysis,  and  mathematical  software. 

The  collaboration  between  the  Applied  and  Computational  Mathematics  Division  and  the 
Materials  Science  and  Engineering  Laboratory  at  NIST  in  materials  modeling  is  mature  and 
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healthy.  Among  the  joint  projects  are  efforts  in  using  phase  field  models  for  dendritic  growth, 
solidification  of  alloys,  contact  lines,  and  isotropic-nematic  interfacial  instability  and  studies  of 
reactive  spreading  of  solder  drops,  directional  solidification  modeling,  stochastic  approaches  to 
problems  in  polymer  science,  viscoelastic  modeling  of  polymers  near  the  glass  transition,  and 
elasticity  and  fracture  of  composite  materials. 

Following  a  recommendation  of  the  1994  Assessment  Panel,  the  division  moved  to 
intensify  its  collaboration  with  the  Materials  Science  and  Engineering  Laboratory  through 
participation  in  the  Center  for  Theoretical  and  Computational  Material  Science.  Eight  members 
of  the  division  are  participants  in  this  center,  although  funding  for  the  center  has  been  limited. 

In  physics  and  chemistry  modeling,  the  projects  dealing  with  modeling  of  epitaxial  growth, 
resonance  and  chaos  in  multifrequency  systems,  and  internal  waves  in  a  viscometer  near  the 
critical  point  are  all  noteworthy.  In  the  area  of  computational  geometry,  the  work  on  large-scale 
surface  representations  and  constrained  mesh  generation  has  made  significant  progress  and  has  led 
to  a  successful  collaboration  with  the  Scientific  Visualization  Group  within  the  Scientific 
Computing  Environments  Division.  An  important  and  relatively  new  NIST  competency  project 
on  the  analysis  and  calibration  of  coordinate  measuring  machines  involves  scientists  from  the 
Applied  and  Computational  Mathematics  Division,  the  Statistical  Engineering  Division,  and  the 
Manufacturing  Engineering  Laboratory.  An  analysis  of  the  probe  error  in  such  machines  has  led 
to  corrections  that  improve  the  overall  measurement  process. 

In  image  analysis,  there  has  been  fijrther  progress  in  finding  more  effective  deblurring 
techniques  for  positron  emission  tomography    This  is  joint  work  with  the  National  Institutes  of 
Health  and  should  find  widespread  acceptance  within  the  medical  community    Other  separate 
activities  in  inverse  problems  are  also  under  way  within  the  division. 

The  division's  work  in  mathematical  algorithms  and  software  has  achieved  significant 
impact  through  the  wide  dissemination  of  the  Guide  to  Available  Mathematical  Software 
(GAMS)    The  number  of  accesses  to  the  GAMS  World  Wide  Web  page  during  the  month  of 
February  reached  its  highest  monthly  total  ever  (8,461  hits)    Public  awareness  of  GAMS  is 
increasing  rapidly  as  evidenced  by  a  recent  front  page  story  in  the  Society  of  Industrial  and 
Applied  Mathematics  Ne\\'s  (Vol.  27,  no  8,  October  1994)  announcements  in  Numerical  Analysis 
Digest,  and  listings  in  several  publications  and  newsletters  serving  users  of  the  World  Wide  Web. 
A  new  project  on  object-oriented  numerical  software  seeks  to  develop  and  make  available 
reusable  software  components  for  important  scientific  computing  applications.  Iterative  solutions 
of  large  linear  systems  and  sparse  matrix  objects  are  two  of  the  collections  of  algorithms 
implemented  in  C++  and  made  available  through  the  World  Wide  Web. 

The  division's  equipment  and  facilities  appear  to  be  adequate.  Computer  hardware  is 
upgraded  regularly,  and  the  current  shortage  of  laboratory  space  should  be  alleviated  by  the  move 
to  NIST  North. 


Assessment  of  Impact  on  Industry 

The  metrics  used  by  division  management  to  evaluate  the  industrial  impact  of  their 
programs  are  based  on  the  frequency  of  use  of  the  division's  software  databases,  the  number  of 
publications  and  citations,  and  the  number  of  industry-oriented  workshops  organized  by  the 
division    Although  it  is  understood  that  the  division's  impact  is  (currently)  of  an  indirect  nature. 
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the  assessment  methodology  is  nonetheless  nonstrategic.  The  division  has  identified  six  strategic 
thrusts,  and  an  assessment  of  its  impact  should  use  criteria  based  on  the  identified  thrusts. 


Recommendations 

The  following  are  the  panel's  recommendations  for  the  Applied  and  Computational 
Mathematics  Division. 

•  The  management  of  the  Applied  and  Computational  Mathematics  Division  should  develop  long- 
term  goals  for  each  thrust  area.  The  statement  of  these  goals  should  be  specific  enough  to  guide 
project  selection  and  should  also  serve  to  identify  impacts  on  customers  and  define  measures  of 
success. 

•  Division  management  should  regularly  review  and  update  program  thrusts  to  identify  high-priority 
research  projects  and  encourage  a  new  focus  on  significant  mathematical  topics  within  the 
information  technology  arena 


Applied  and  Computational  Mathematics  Division  Responses  to  Fiscal  Year  1994 
Recommendations 

Below  are  the  panel's  1994  recommendations  for  the  Applied  and  Computational 
Mathematics  Division,  quoted  from  the  fiscal  year  1994  report,  followed  by  the  division's 
response. 

1994  Panel  Recommendalion:  ".    .  [Division]  planning  should  address  the  fact  that  its  goals  are 
too  ambitious  for  its  current  size  and  structure"  (p  241). 

Division  Response:  Division  goals  are  indeed  ambitious    The  goals  are  currently  being  revised  in 
the  context  of  planning  for  the  new  Information  Technology  Laboratory. 

1994  Panel  Recommendation:  "The  [division]  should  continue  to  emphasize  the  formulation  of  a 
specific  team  project  in  mathematics-based  product  design  and  manufacturing  .  .  ."  (p.  241) 
Division  Response:  The  division  has  begun  participation  in  a  joint  CAML/Manufacturing 
Engineering  Laboratory  effort  in  the  computational  metrology  of  manufactured  parts. 

J994  Panel  Recommendation:  "The  [division]  should  be  proactive  in  communicating  to  NIST 
management  the  role  and  importance  of  applied  and  computational  mathematics  in  NIST's 
interdisciplinary  research  programs    The  division's  leaders  should  maintain  a  list  of 
accomplishments  that  have  had  a  significant  impact  on  other  NIST  operations  or  on  specific 
industrial  practices    Practical  impacts  should  be  emphasized"  (p  241) 

Division  Response:  The  division  has  emphasized  this  activity  in  the  past  year.  The  division  chief 
wrote  a  memorandum  to  the  NIST  director  describing  the  importance  of  applied  and 
computational  mathematics  and  the  division's  role  in  fostering  interdisciplinary  research.  The 
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division  hosted  a  laboratory  visit  for  the  NIST  assistant  director  to  provide  an  in-depth  look  at  a 
selection  of  projects. 

1994  Panel  Recommendation:  "The  [division]  should  increase  promotion  and  dissemination  of 
mathematical  and  computational  tools  developed  by  the  division"  (p.  241). 
Division  Response:  The  division  has  emphasized  this  activity  in  the  past  year.  The  Guide  to 
Available  Mathematical  Software  was  formally  launched  with  overwhelming  response.  In  early 
1995,  inquiries  to  GAMS  accounted  for  more  than  one-half  of  all  World  Wide  Web  traffic  into 
NIST.  In  addition,  an  anonymous  FTP  (file  transfer  protocol)  site  has  been  established  to  provide 
alternate  access  to  this  material. 

1994  Panel  Recommendation:  "The  [division]  should  actively  promote  and  reward  participation 
in  the  planned  MST  Industry  Fellows  Program  and  use  this  as  a  means  to  establish  direct  links  to 
industry.  .  .  ."  (p.  241). 

Division  Response:  The  Industry  Fellows  Program  does  indeed  provide  an  excellent  mechanism 
for  establishing  links  to  industry,  and  the  division  will  continue  to  explore  such  possibilities. 
However,  it  has  been  difficult  to  motivate  staff  to  participate  and  to  find  ideal  industrial  partners. 

1994  Panel  Recommendation:  "The  [division]  should  intensify  its  collaboration  with  MSEL 
[Materials  Science  and  Engineering  Laboratory]  through  the  Distributed  Center  for  Theory  and 
Modeling  of  Materials  currently  being  planned"  (p.  242). 

Division  Response:  The  proposed  center  is  now  in  operation  as  the  Center  for  Theoretical  and 
Computational  Materials  Science,  but  fiinding  is  minimal.  Eight  division  staff  members  are 
registered  participants  in  the  center.  An  increase  in  division  participation  requires  greater  funding 
from  the  center. 


Statistical  Engineering  Division 

Introduction 

Despite  the  uncertainty  wrought  by  the  imminent  merger  of  the  Computing  and  Applied 
Mathematics  Laboratory  and  the  Computer  Systems  Laboratory,  by  the  move  to  NIST  North,  and 
by  the  lack  of  a  permanent  division  director,  the  Statistical  Engineering  Division  has  had  a 
productive  and  high-quality  year.  Furthermore,  the  staffs  morale  and  enthusiasm  for  their  science 
are  remarkably  high    Much  of  the  credit  for  this  positive  situation  must  go  to  the  able  leadership 
of  the  acting  division  chief  and  to  the  division  staff. 


Mission 

The  Statistical  Engineering  Division  seeks  to  catalyze  scientific  and  industrial  research 
through  the  application  of  statistical  methods  to  the  experimentation  and  data  analysis  underlying 
the  empirical  information  garhering  processes  critical  to  NIST  scientists  and  engineers. 
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Resources 


Fiscal  year  1994  funding  for  the  division  totaled  $3.3  million    Of  these  funds,  80  percent 
came  directly  from  SIRS  with  the  balance  coming  from  nonbase  STRS  funds  and  NIST 
interdivision  transfers.  The  division  employs  25.4  full-time  equivalent  staff,  of  whom  22  are 
technical  professionals. 


The  Priority-Setting  Process 

The  Statistical  Engineering  Division  works  in  areas  where  statistical  expertise  in 
consulting  and  collaborative  research  has  the  potential  to  contribute  significantly  to  the  NIST 
mission  by  supporting  fundamental  metrology,  calibration  and  SRMs  programs,  outreach 
programs  designed  to  transfer  measurement  technology  to  industry,  and  high-profile  projects  of 
national  interest.  The  new  emphasis  at  NIST  on  developing  infrastructural  methods  that  directly 
drive  technology  has  led  the  division  to  expand  its  traditional  mission  to  include  support  for  the 
national  information  infrastructure  via  the  High-Performance  Computing  and  Communications 
initiative,  establishment  of  direct  relationships  with  industrial  users,  and  contributions  to  NIST's 
extramural  programs. 

To  remain  on  the  cutting  edge  of  research,  keep  abreast  of  important  new  trends,  and 
generate  a  solid  and  credible  research  product,  the  division  maintains  active  links  with  strong 
academic  and  industrial  research  groups.   In  fiscal  year  1995,  the  division  hosted  nine  guest 
researchers,  two  adjunct  faculty,  two  postdoctoral  researchers,  and  numerous  invited  speakers. 


Assessment  of  Priorities  and  the  Priority-Setting  Process 

The  division's  primary  mode  of  operation  is  to  respond  to  requests  for  collaboration  from 
other  NIST  scientists  and  to  mandates  from  Congress  and  the  Executive  Branch.  Its  mission 
statement  casts  the  division  as  a  service  unit  to  the  rest  of  the  NIST  enterprise,  and  to  this  extent, 
its  priorities  are  influenced  by  those  of  the  other  laboratories    When  the  demand  for  assistance 
exceeds  the  division's  ability  to  deliver,  the  directors  of  divisions  that  have  multiple  projects  are 
asked  to  sharpen  their  prioritization  to  more  effectively  meet  NIST  needs.  Although  the 
advancement  of  statistical  science  is  not  explicitly  included  in  its  mission  statement,  the  division 
has  played  and  continues  to  play  a  significant  role  in  defining  and  advancing  such  research.  The 
Senior  Research  Fellowship  Program,  which  brings  in  leading  experts  from  industry  and  academia 
for  extended  sabbaticals,  is  an  innovative  source  of  input  in  setting  priorities. 

In  the  context  of  its  service  mission,  the  division's  commitment  of  about  40  percent  of  its 
effort  to  the  methodology  of  measurement  science,  calibration,  and  SRMs  is  appropriate  in  that  it 
is  fundamental  to  the  NIST  mission    The  division's  work  adds  great  value  to  NIST's  role  in 
standards  setting.  In  addition,  the  division  responds  promptly  and  appropriately  to  special 
projects  mandated  by  Congress  and  by  Executive  Order. 

Even  though  the  division's  consulting  and  collaboration  with  other  NIST  laboratories  are 
funded  directly  by  STRS,  the  division  takes  its  customers  seriously.  Although  the  division  has 
collaborated  with  every  NIST  laboratory,  the  distribution  of  collaborations  is  far  from  uniform 
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across  the  institute.  Generally,  the  choice  of  collaborative  projects  is  driven  by  individual  contacts 
with  researchers  in  other  laboratories.  Little  guidance  has  been  provided  from  above,  although 
this  is  not  uncommon  for  a  statistical  consulting  group.  Notably,  the  one  laboratory  with  which 
the  division  has  had  almost  no  interaction  is  the  Computer  Systems  Laboratory.  Opportunities  for 
collaboration  could  be  found  in  establishing  metrics  and  standards  for  building  and  testing 
software  and  in  assessing  network  performance. 


Assessment  of  Technical  Programs 

The  technical  programs  pursued  within  the  Statistical  Engineering  Division  continue  to  be 
of  the  highest  quality.  An  estimated  80  percent  of  the  work  is  done  in  collaboration  with  other 
NIST  scientists  and  is  widely  distributed  across  the  NIST  laboratories.  The  fact  that  these 
laboratories  continue  to  seek  out  collaborations  with  the  Statistical  Engineering  Division  is 
testimony  to  the  quality  of  the  division's  work.  Of  particular  note  are  examples  in  which  division 
researchers  have  advanced  the  statistical  contributions  to  these  collaborations  beyond  the  original 
perceived  need  of  the  collaborators.  In  addition  to  the  enhancement  of  the  specific  project,  such 
initiatives  introduce  the  collaborating  scientists  from  other  laboratories  to  the  power  and  breadth 
of  statistics  and  to  new  (at  least  to  them)  statistical  methods.  In  other  words,  these  initiatives 
effect  a  transfer  of  statistical  technology  from  the  Statistical  Engineering  Division  to  other 
laboratories  and  divisions  within  NIST. 

With  regard  to  the  status  of  equipment  and  facilities,  a  serious  space  shortage  at  Boulder 
threatens  the  division's  ability  to  appoint  faculty  and  fellows.  This  is  a  laboratory-level  problem 
requiring  laboratory-level  intervention.  The  second  facility  concern  is  the  impact  of  moving  to 
NIST  North  on  the  spontaneity  of  face-to-face  collaboration  and  consultation.  The  distance  and 
parking  problems,  although  surmountable,  will  discourage  the  easy  exchange  of  ideas.  However, 
the  problems  offer  opportunities  to  establish  a  testbed  for  videoconferencing  and  other  site-to-site 
communication  technologies 


Assessment  of  Impact  on  Industry 

The  impact  of  the  Statistical  Engineering  Division  on  other  NIST  laboratories  and  on 
industry  is  deep  and  extensive.  The  division's  work  on  SRMs,  calibration,  and  metrology  adds 
tremendous  value  to  NIST  products.  In  effect,  the  division  sets  the  standard  for  the  setting  of 
standards.  The  concomitant  effect  on  industry  is  ubiquitous.  A  recent  example  is  the 
development  of  an  on-line  statistics  manual  with  integrated  software  for  the  electronics  industry  in 
collaboration  with  SEMATECH.  The  range  of  collaborations  with  other  NIST  laboratories  that 
eventually  lead  to  industrial  applications  is  very  wide. 


Recommendations 

The  following  are  the  panel's  recommendations  for  the  Statistical  Engineering  Division. 
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The  essential  role  of  the  Statistical  Engineering  Division  in  support  of  metrology,  calibration,  and 
SRMs  must  be  maintained  no  matter  what  the  ultimate  structure  of  the  Information  Technology 
Laboratory. 

The  division  staff  should  explore  opportunities  for  statistics  in  network  modeling  and  analysis, 
software  engineering,  and  other  applications  to  information  technology.  The  division  should  add 
staff  with  relevant  expertise  if  necessary. 

Despite  the  excellent  work  of  the  current  acting  division  chief  a  permanent  chief  should  be 
appointed  as  quickly  as  possible. 


Statistical  Engineering  Division  Responses  to  Fiscal  Year  1994  Recommendations 

Below  are  the  panel's  1994  recommendations  for  the  Statistical  Engineering  Division, 
quoted  from  the  fiscal  year  1994  report,  followed  by  the  division's  response 

1994  Panel  Recommendation:  "CAML  should  charge  an  interim  division  chief ...  to 
aggressively  shape  the  division's  fliture  in  the  new  ITL"  (p.  237) 

Division  Response:  An  acting  division  chief  was  appointed  during  the  past  year. 

J994  Panel  Recommendation:  "The  [division]  should  monitor  its  selection  of  projects  to  ensure 

that  it  focuses  its  effort  on  those  most  central  to  the  NIST  mission"  (p.  238). 

Division  Response:  Although  the  division  relies  primarily  on  informal  management  involvement 

in  project  selection,  there  is  an  established  process  that  brings  projects  to  the  division. 

J  995  Panel  Comment:  A  large  part  of  the  work  of  the  division  focuses  on  fundamental  research 

in  measurement  science,  calibration,  SRMs,  and  the  transfer  of  measurement  technology,  which 

are  all  technical  aspects  of  the  metrology  mission    The  panel  commends  the  division  on  its 

support  of  this  fundamental  mission.  The  recent  hiring  of  new  staff  with  series  expertise  also 

reflects  a  responsiveness  to  changing  priorities 

J994  Panel  Recommendation:  "In  developing  a  project  management  system,    .  .  the  [division] 

should  consider  establishing  liaison  relationships  organized  around  high-priority  themes" 

(p.  238). 

Division  Response:  The  division  has  established  liaisons  organized  around  high-priority  technical 

programs,  with  the  exception  of  collaborative  research  that  generally  flows  from  contacts  with 

individual  scientists. 

1995  Panel  Comment:  The  division  has  established  some  specialization  within  the  staff  For 
example,  the  traditional  work  in  calibration  and  SRMs  is  concentrated  in  a  few  people.  It  has 
additional  connections  in  high-performance  computing  and  communications.  However,  the 
division  continues  to  feel  that  a  strong  thematic  substructure  is  incompatible  with  a  flexible  and 
responsive  consulting  organization. 

199-1  Panel  Recommendation:  "Division  .  .  .  management  should  ensure  that  upper 
administration  is  regulariy  apprised  of  the  division's  outreach"  (p.  238). 
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Division  Response:  During  the  past  year,  the  division  management  initiated  discussions  on  the 
proper  role  of  the  division  with  the  MST  deputy  director,  the  deputy  director  of  the 
Manufacturing  Engineering  Laboratory,  and  the  director  of  Technology  Services.  In  addition,  the 
NTST  deputy  director  made  two  visits  to  the  division's  laboratories  to  learn  about  current 
projects. 

1995  Panel  Comment:  The  division  has  been  putting  more  effort  into  communicating  the  breadth 
and  depth  of  its  contributions  in  the  1995  Report  of  Activities.  On  a  day-to-day  basis,  staff  have 
initiated  several  meetings  with  the  NIST  deputy  director  and  the  management  of  other 
organizations  within  >nST  to  explain  their  activities  and  contributions 


Information  Technology  Services 
Introduction 

Information  Technology  Ser\'ices  provides  NIST  with  central  scientific  computing 
services,  network  and  data  communications,  software,  voice  communications,  administrative 
computing  services,  and  related  scientific  and  technical  support    Four  laboratory  divisions  are 
involved  in  carrying  out  these  responsibilities:  the  Scientific  Computing  Environments  Division, 
which  is  responsible  for  scientific  visualization,  user  training,  and  software  administration  and 
support;  the  Computer  Services  Division,  which  is  responsible  for  management,  operation,  and 
administration  of  central  computing  resources,  the  Computer  Systems  and  Communications 
Division,  which  is  responsible  for  central  networking  and  telecommunications  services  and  major 
systems  acquisitions;  and  the  Information  Systems  Division,  which  is  responsible  for 
administrative  computing  support 

The  Office  of  the  Associate  Director  for  Computing  (ADC)  is  responsible  for  the  overall 
planning,  coordination,  and  policy  direction  of  the  resources  and  operations  of  the  NIST  Scientific 
Computing  Facility  and  the  activities  and  resources  associated  with  central  scientific  computing 
and  communications  services    This  includes  assessment  of  the  computing  needs  of  NIST 
scientific  programs,  assessment  of  current  trends  in  information  technology,  planning  for  and 
installing  new  computer  hardware,  and  design  of  software  and  communications  strategies    The 
Office  of  the  Associate  Director  for  Computing  coordinates  the  development  and  dissemination  of 
scientific  software  for  the  high-performance  computing  systems  component  of  the  NIST  High- 
Performance  Computing  and  Communications  initiative  and  is  also  responsible  for  network 
security. 


Mission 

The  mission  of  those  divisions  that  constitute  Information  Technology  Services  is  to 
deliver  the  computation,  communication,  and  information  technology  resources  necessary  to 
provide  NIST  laboratories  and  their  collaborative  partners  with  a  strong  foundation  for  state-of- 
the-art  science. 
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Resources 

The  staff  of  the  Scientific  Computing  Environments  Division  consists  of  31  technical 
professionals  and  1  administrative  support  person    The  division's  total  budget  for  fiscal  year  1994 
was  $2.7  million. 

The  staff  of  the  Computer  Services  Division  consists  of  20  technical  professionals  and  1 
administrative  support  person.  The  division's  total  budget  for  fiscal  year  1994  was  $2.9  million. 

The  staff  of  the  Computer  Systems  and  Communications  Division  consists  of  32  technical 
professionals  and  10  contractors.  The  division's  total  budget  for  fiscal  year  1994  was  $7.9 
million 

The  staff  of  the  Information  Systems  Division  consists  of  33  technical  professionals  and  3 
support  personnel    The  division's  total  budget  for  fiscal  year  1994  was  $2.8  million. 


The  Priority-Setting  Process 

The  major  priority  for  the  Scientific  Computing  Environments  Division  is  to  provide 
central  support  for  the  current  Computing  and  Applied  Mathematics  Laboratory,  as  well  as 
consultation  and  advice  for  NIST  users  of  central  computing  facilities    New  projects  are 
undertaken  af\er  user  needs  are  assessed,  and  an  effort  is  made  to  implement  new  technologies 
that  add  capability  to  NIST's  computing  facilities    The  division  occasionally  receives  specific 
direction  from  the  office  of  the  NIST  director. 

The  first  priority  of  the  Computer  Services  Division  is  to  ensure  the  availability  of  Central 
Computing  Facility  resources  to  the  NIST  user  community.  New  projects  are  undertaken  to 
improve  existing  services  or  to  add  new  services  and  result  from  a  continuous  review  process 
carried  out  in  conjunction  with  the  Office  of  the  Associate  Director  for  Computing. 

The  priorities  of  the  Computer  Systems  and  Communications  Division  revolve  around  the 
division's  effort  to  provide  reliable  service  and  leading-edge  technology.  These  somewhat 
incompatible  goals  are  determined  by  its  customers,  some  of  whom  want  100  percent  up-time  and 
others  of  whom  want  the  latest  prerelease  versions  of  hardware  and  software.  The  division  strives 
to  balance  these  two  goals,  giving  slightly  greater  weight  to  reliability 


Assessment  of  Priorities  and  the  Priority-Setting  Process 

The  Office  of  the  Associate  Director  for  Computing  has  implemented  an  explicit  and  well- 
defined  planning  process.  The  process  includes  developing  annual  acquisition  plans  for  the 
Department  of  Commerce  and  involves  significant  input  from  its  NIST  customers  gathered 
through  management  discussions,  interaction  with  the  Scientific  Computing  Planning  Group,  and 
feedback  from  users    The  selection  of  priorities  is  based  on  several  factors,  including  staff 
competence  and  an  assessment  of  which  projects  would  bring  the  greatest  benefit  to  the  largest 
number  of  users.  Priorities  are  also  greatly  affected  by  mandates  from  NIST  or  Department  of 
Commerce  management.   Such  mandates  can  subvert  the  priority-setting  process  and  often 
require  inordinate  amounts  of  ADC  management  attention. 
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One  sign  of  the  effectiveness  of  the  priority-setting  process  is  that  the  office  can  articulate 
examples  of  activities  or  services  that  it  has  stopped  providing  and  examples  of  certain  activities 
or  services  that  it  has  explicitly  chosen  not  to  undertake.  These  decisions  are  based  on  customer 
input  and  resource  competition.  Examples  of  services  no  longer  provided  include  terminal  server 
and  modem  network  support. 

Although  program  planning  appears  to  be  effective  and  the  selection  of  projects  is 
appropriate,  the  ADC  should  consider  employing  more  rigorous  planning  methods  to  manage 
those  projects  directed  toward  the  development  and  installation  of  computing  and 
communications  facilities.  Such  projects  include  processor  acquisitions,  upgrades  and 
installations,  large-scale  network  design,  and  implementation  projects  such  as  those  under  way  for 
the  new  buildings. 


Assessment  of  Technical  Programs 

In  general.  Information  Technology  Services  provides  NIST  scientists  and  staff  with  a 
high-quality  computing  and  communication  environment  that  is  close  to  state  of  the  art  in  most 
areas,  including  the  use  of  asynchronous  transfer  mode  (ATM)  network  technology,  processor 
capability,  visualization,  parallel  application  support,  and  data  storage    The  development  of 
parallel  application  development  support  tools  and  the  transfer  of  this  technology  to  industry  are 
particularly  noteworthy.  The  efficient  manner  in  which  the  dramatic  increase  in  networking  and 
communication  needs  was  met  without  increasing  the  number  of  staff,  and  the  movement  of  the 
visualization  technology  effort  from  lagging  to  near  state  of  the  art  in  a  relatively  short  period  of 
time,  are  both  examples  of  outstanding  achievement. 

The  panel  is  concerned  about  the  impact  of  the  Commerce  Administrative  Management 
System  project  on  the  ADC.  This  is  a  large-scale,  client-server  application  with  a  very  large 
number  of  potential  clients  and  is  possibly  global  in  scope.  The  ADC  has  little  experience  with 
this  technology,  and  there  is  an  issue  with  the  robustness  of  the  software.  It  is  not  clear  that  the 
ADC  is  adequately  prepared  to  address  the  myriad  issues  that  will  arise.  The  project  would 
benefit  by  the  adoption  of  a  client-server  application  development  methodology,  including  a  much 
more  rigorous  planning  and  testing  process. 


Assessment  of  the  Impact  on  Industry 

The  Office  of  the  Associate  Director  for  Computing  and  the  four  divisions  of  the 
Information  Technology  Laboratory  that  together  constitute  Information  Technology  Services 
work  primarily  with  other  divisions  and  laboratories  within  NIST,  and  the  industrial  impact  of 
their  activities  is  necessarily  indirect.  However,  ADC  provides  direct  feedback  to  vendors  on 
product  direction  and  needs  and  is  involved  in  providing  review  and  consultation  to  the  Advanced 
Technology  Program  (ATP).  One  novel  area  of  industry  interaction  is  software  transfer  to 
independent  software  vendors  through  the  Small  Business  Innovative  Research  Grant  Program. 
In  the  parallel  application  development  area,  ADC  has  made  a  concerted  effort  to  transfer 
development  products  to  non-NIST  scientists  through  the  Internet  and  through  conference 
presentations. 


19 


34 


Recommendations 


The  following  are  the  panel's  recommendations  for  the  Office  of  the  Associate  Director 
for  Computing. 

The  panel  urges  the  Office  of  the  Associate  Director  for  Computing  to  expand  the  use  of  external 
contractors  to  off-load  routine  services,  such  as  PC  maintenance  and  support,  trouble-desk 
staffing,  support  of  distributed  servers,  and  support  of  local  area  networks. 

The  panel  recommends  that  the  Office  of  the  Associate  Director  for  Computing  maintain  its  focus 
on  the  distributed  computing  environment  to  ensure  that  as  the  technology  becomes  available,  the 
ADC  is  in  a  position  to  implement  it. 


Information  Technology  Services  Divisions  Responses  to  Fiscal  Year  1994 
Recommendations 

Below  are  the  panel's  1994  recommendations  for  the  Information  Technology  Services 
Division,  quoted  from  the  fiscal  year  1994  report,  followed  by  the  division's  response. 

1994  Panel  Recommendation:  ".  .  .  CAML  staff  [should]  investigate  the  requirements  for  and 
plan  a  network  security  program"  (p.  244). 

Division  Response:  Network  engineering  groups  in  both  Gaithersburg  and  Boulder  are  installing 
small  firewalls  in  the  laboratories  in  order  to  learn  more  about  these  systems  and  to  better  support 
the  efforts  of  the  Computer  Security  Division. 

1995  Panel  Comment:  The  laboratory  has  effectively  responded  to  the  recommendation  of  last 
year's  panel  by  designing  and  implementing  firewalls  for  network  security. 

1994  Panel  Recommendation:  "An  allocation  committee  should  be  created  consisting  of 
representative  users  from  across  NIST  charged  with  the  responsibility  of  making 
recommendations  for  resource  allocations  based  on  the  technical  strength  of  proposals  from  the 
user  community"  (p.  244). 

Division  Response:  A  committee  has  been  formed  to  oversee  all  aspects  of  the  IBM  SP-2 
including  user  access.  The  Cray  was  purchased  with  central  computing  funds  and  is  intended  to 
be  a  general  resource  available  to  all  NIST  laboratories.  The  capabilities  of  the  Cray  have  been 
upgraded  such  that  regulation  of  use  is  not  required  at  this  time. 

1995  Panel  Comment:  The  panel  predicts  that  the  upgraded  Cray  will  soon  be  saturated,  and  to 
maximize  the  scientific  utility  of  the  machine,  some  control  should  be  exerted  on  its  use. 

1994  Panel  Recommendation:  "CAML  should  consider  adding  a  PhD-level  researcher  in 
computational  science  with  expertise  in  parallel  computing  to  fully  utilize  the  SP-2"  (p.  244). 
Division  Response:  The  division  has  hired  an  individual  with  the  requisite  expertise  to  work  on 
software  for  the  SP-2. 
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Systems  and  Network  Architecture  Division 


Mission 


The  division  facilitates  the  development  and  use  of  networking  technology  through  the 
development  of  networking  standards,  testing  tools  and  methodologies  that  facilitate 
interoperability,  and  activities  that  demonstrate  state-of-the-art  technology. 


Resources 

Fiscal  year  1994  funding  for  the  division  totaled  $5.5  million    Of  the  total,  $2.4  million 
are  from  STRS  and  $3.1  million  from  reimbursable  services  such  as  calibration  services  and  the 
sale  of  SRMs.  Division  staff  consists  of  37  fliil-time-equivalent  employees,  of  whom  27  are 
technical  professionals. 


The  Priority-Setting  Process 

Through  its  interactions  with  industry,  government,  and  other  NIST  laboratories,  the 
division  identifies  areas  of  need  and  evaluates  whether  it  has  the  resources  and  capabilities  to 
make  a  unique  contribution  toward  addressing  those  needs    Division  management  applies  the 
following  criteria  to  all  potential  projects: 

•  How  well  does  the  project  fit  the  division's  mission? 

•  What  is  the  expected  payoff  to  the  US.  economy? 

•  What  level  of  risk  is  involved? 

•  Who  are  the  potential  partners  and  clients,  and  what  is  their  level  of  commitment 
and/or  need? 


Assessment  of  Priorities  and  the  Priority-Setting  Process 

Division  management  understands  that  the  division's  strengths  lie  in  network  protocols 
and  electronic  commerce  and  has  assigned  two-thirds  of  the  technical  staff  to  these  areas.  These 
areas  are  aligned  with  trends  in  information  technology  and  industry  needs.  The  division  has  a 
good  sense  of  direction,  but  it  does  not  have  an  explicit  process  for  setting  priorities.  Priorities 
are  currently  established  in  an  opportunistic  rather  than  a  strategic  manner. 


Assessment  of  Technical  Programs 

Division  programs  support  network  infrastnicture  (eg,  routing  and  forwarding  protocols, 
management  protocols,  security  protocols),  result  in  useful  applications  (e.g.,  electronic  mail, 
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directory  services,  electronic  commerce),  and  are  needed  to  support  activities  not  pursued 
sufficiently  elsewhere  (e.g.,  protocol  testing  and  validation,  performance  studies). 

Efforts  supporting  Internet  protocols  and  work  on  the  electronic  commerce  initiative  are 
noteworthy,  particularly  the  use  of  prototypes  to  demonstrate  interoperability  and  integration  of 
applications.  On  the  other  hand,  the  effort  in  formal  methods  is  not  adequately  staffed  to  be 
effective. 

The  division  is  engaged  in  a  successful  program  to  develop  improved  network  protocols 
but  does  not  appear  to  coordinate  its  efforts  with  the  research  conducted  in  the  Advanced 
Systems  Division  on  ATM  and  wireless  networking. 


Assessment  of  Impact  on  Industry 

The  Electronic  Commerce  Integration  Facility  has  immediate  impact  on  industry,  and 
expansion  of  this  program  is  warranted    Both  the  number  and  breadth  of  collaborations  are 
impressive    For  example,  25  vendors  contributed  products  to  the  electronic  commerce  testbed 


Recommendations 

The  following  are  the  panel's  recommendations  for  the  Systems  and  Network  Architecture 
Division. 

The  management  of  the  Systems  and  Network  Architecture  Division  has  identified  the  needed 
transition  into  the  future,  but  programs  require  focus  and  a  proactive  plan  for  facilitating  the 
transition. 

The  division  should  coordinate  its  network  protocol  effort  with  the  Advanced  Systems  Division's 
efforts  in  asynchronous  transfer  mode  and  wireless. 


Systems  and  Network  Architecture  Division  Responses  to  Fiscal  Year  1994 
Recommendations 

Below  are  the  panel's  1994  recommendations  for  the  Systems  and  Network  Architecture 
Division,  quoted  from  the  fiscal  year  1994  report,  followed  by  the  division's  response. 

1994  Panel  Recommendation:  "The  [division's]  planning  document  should  be  recast  in  terms  of 
the  Nil  and  electronic  commerce  so  that  the  role  and  responsibility  of  the  division  are  clear  and  its 
projects  can  be  clearly  identified  with  these  national  initiatives.  The  division's  role  in  support  of 
the  ATP  should  also  be  specified  in  the  strategic  plan"  (p.  219). 

Division  Response:  The  division  is  aware  that  its  original  strategy  document  is  out  of  date.  The 
division  has  decided  to  produce  a  minidocument  that  will  later  be  included  in  a  full-fledged 
strategy  document  once  the  reorganization  of  the  laboratory  is  complete. 
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1994  Panel  Recommendation:  "Division  projects  on  the  Nil  and  electronic  commerce  should 
focus  on  interoperability  testing,  perhaps  using  the  Electronic  Commerce  Integration  Facility  or 
some  similar  user-perspective  basis  on  which  to  test  the  adequacy  and  usability  of  standards  and 
prototypes"  (p.  219). 

Division  Response:  The  division  has  initiated  work  toward  the  development  of  testing  materials 
and  plans  to  expand  this  activity. 

1994  Panel  Recommendation:  "The  investigation  into  secure  electronic  mail  needs  to  sharpen  its 
goals  and  objectives"  (p.  219). 

Division  Response:  The  division  is  negotiating  a  Cooperative  Research  and  Development 
Agreement  (CRADA)  with  companies  that  intend  to  build  office  architectures  around  electronic 
mail  and  directories. 

1995  Panel  Comment:  By  working  with  industry  on  a  CRADA,  the  division  will  likely  achieve  a 
sharper  focus  with  respect  to  its  goals  and  objectives  in  the  area  of  secure  electronic  mail. 

1994  Panel  Recommendation:  "The  division's  mission  statement  needs  to  be  analyzed, 
consolidated,  and  made  more  visible    The  current  mission  statement  is  not  useful  in  focusing  the 
strategy"  (p.  219). 

Division  Response:  The  division  may  not  be  able  to  meet  this  recommendation  fully  within  the 
current  reorganization  environment. 

1995  Panel  Comment:  The  division  is  waiting  to  respond  until  a  director  for  the  Information 
Technology  Laboratory  is  in  place. 

1994  Panel  Recommendation:  "Laboratory  facilities  [NIST  North]  must  be  planned  so  as  not  to 
disrupt  the  division's  pending  network  connection,  even  if  this  means  keeping  the  division  on  the 
main  campus  while  other  divisions  move  off-campus"  (p  219). 

Division  Response:  The  division  is  planning  for  the  move  to  NIST  North  with  a  view  of  not  only 
maintaining  existing  facilities  but  upgrading  them  as  well 

1995  Panel  Comment:  The  division  has  responded  well  to  this  recommendation. 


Computer  Security  Division 

Mission 

The  mission  of  the  division  is  to  enable  organizations  and  individuals  to  use  information 
technology  with  the  assurance  and  trust  that  the  confidentiality,  integrity,  reliability,  and 
availability  of  information  resources  are  protected. 

Resources 

The  division  has  36  full-time  employees  and  4  part-time  employees.  Guest  researchers  and 
facility  members  work  with  division  staff  on  selected  projects    Fiscal  year  1995  estimated  flinding 
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is  $5.4  million  from  STRS,  $1.6  million  from  reimbursable  flinds,  and  $186,000  from  the 
Information  Technology  Services  (ITS)  initiative. 


The  Priority-Setting  Process 

Since  it  is  beneficial  for  both  government  and  industry  to  employ  interoperable  information 
technology,  both  the  private  and  public  sectors  are  considered  in  the  priority-setting  process. 
Priorities  are  set  by  obtaining  direction  and  support  from  developers,  integrators,  and  users  who 
are  concerned  with  the  protection  of  information  technology  resources    Specific  examples  include 
security  technology  developers  and  vendors,  private  sector  organizations,  and  government 
agencies.  Input  from  these  users  and  purveyors  of  information  technology  gathered  through 
workshops,  seminars,  and  collaborations  is  given  due  consideration  during  the  priority-setting 
process,  which  involves  both  division  management  and  research  staff 


Assessment  of  Priorities  and  the  Priority-Setting  Process 

The  priorities  and  priority-setting  process  within  the  division  are  well  developed  and  are  in 
keeping  with  its  mission  statement  and  program  management  strategy    A  significant  change  in  the 
initiation  of  projects  based  on  program  thrusts  has  taken  place    The  division  has  begun  projects 
within  each  program  that  capitalize  on  priorities,  resource  leverage,  and  outreach. 


Assessment  of  Technical  Programs 

The  division's  focus  has  been  adjusted  to  include  emphasis  on  industry  needs  and  direct 
support  of  NlST's  internal  systems  The  programs  emphasize  enabling  technologies  and 
infrastructure  projects    Attention  has  also  been  given  to  eliminating  or  obtaining  external  support 
for  those  areas  that  are  resource  intensive  (eg,  training,  management  guidance,  risk  reviews) 
The  five  program  areas — cryptography,  common  authentication  architecture,  criteria  and 
assurance,  public  key  infrastructure,  and  Internet  security — directly  support  current  information 
security  issues  and  strategies. 


Assessment  of  Impact  on  Industry 

The  focus  on  US  industry,  coupled  with  the  experience  gained  by  the  support  of  the 
implementation  of  security  measures  within  the  NIST  systems,  ensures  that  the  overall  program 
will  have  a  substantial  impact  on  industry.  The  following  projects  are  of  significant  impact: 

The  Validation  Program  and  the  transfer  of  validation  activities  to  commercial  laboratories, 
The  work  with  private  and  government  sectors  on  Cryptographic  Application  Program  interfaces. 
Development  of  a  high-level  framework  and  supporting  standards  to  improve  interoperability  of 
authentication  technologies  and  services. 
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•  Establishment  of  CRADAs  with  industry  to  address  public  key  infrastructure  interoperability 
issues; 

•  Support  for  the  Advanced  Systems  Division  in  wireless  transmission  errors  with  respect  to 
encrypted  data; 

•  Development  of  Internet  security  tools,  policy  guidance,  security  administration,  and  security 
plan;  and 

•  Completion  of  the  common  criteria  and  investigation  of  alternative  assurance  approaches  to 
current  evaluation  methods. 


Recommendation 

The  following  is  the  panel's  recommendation  for  the  Computer  Security  Division. 

Nine  specific  measures  of  success  have  been  established  by  the  Computer  Security  Division  to 
reflect  the  expected  results  of  a  successfijl  information  security  program    However,  metrics  for 
each  of  the  nine  measures  should  be  established  to  adequately  gauge  their  status.  An  annual 
summary  of  these  metrics  should  be  formally  reported    Also,  at  that  time,  an  internal  assessment 
of  the  effectiveness  of  each  of  the  metrics  should  be  conducted  and  adjustments  made 
accordingly. 

Computer  Security  Division  Responses  to  Fiscal  Year  1994  Recommendations 

Below  are  the  panel's  1994  recommendations  for  the  Computer  Security  Division,  quoted 
from  the  fiscal  year  1994  report,  followed  by  the  division's  response. 

1994  Panel  Recommendation:  "The  [division]  should  develop  a  crisper  and  more  complete 
strategy  statement  to  support  [its]  mission,  which  should  also  be  consistently  stated  in  all  strategic 
planning  documents  and  presentations"  (p.  216  ). 

Division  Response:  The  division  has  developed  a  specific  strategic  goals  statement  as  part  of  the 
Information  Technology  Laboratory  planning  effort. 

1994  Panel  Recommendation:  "The  [division]  should  use  program  plans  to  track  progress  and 
deliverables"  (p.  216). 

Division  Response:  The  division  has  developed  program  and  project  plans  for  all  major  activities; 
they  are  used  to  monitor  the  work. 

1994  Panel  Recommendation:  "The  division  should  consider  assigning  and  increasing  resources 
to  programs  having  broad-based  applicability,  such  as  distributed  authentication  and  Internet 
firewalls"  (p.  216). 

Division  Response:  The  division  is  currently  focusing  special  efforts  on  both  distributed 
authentication  and  Internet  firewalls,  along  with  a  range  of  other  related  Internet  security  work. 

1995  Panel  Comment:  The  Computer  Security  Division's  1995  planning  document  indicates 
close  attention  to  the  1994  recommendations.  With  only  a  slight  deviation  (virus  prevention  and 
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detection),  all  1994  recommendations  have  been  addressed  and  positive  actions  have  been  taken. 
Of  particular  note  is  the  division's  attention  to  project  management  and  tracking  of  actual  versus 
planned  deliverable  dates  for  milestones. 


Systems  and  Software  Technology  Division 

Mission 

The  mission  of  the  Systems  and  Software  Technology  Division  is  to  support  users  and 
industry  through  promotion  and  evolution  of  open  systems  concepts  and  technology,  quality 
software  development  and  integration,  software  engineering  concepts,  document  concepts  and 
technology,  and  heterogeneous  distributed  systems  concepts  and  technology. 

Resources 

The  division  has  38  full-time  employees    Of  the  $4.5  million  in  total  ftjnding  for  fiscal  year 
1994,  $2.5  million  was  from  STRS,  $1.9  million  from  OA  and  reimbursable  services,  and  $0.8 
million  from  the  ITS  initiative 

The  Priority-Setting  Process 

Program  priorities  are  established  by  assessing  a  composite  set  of  issues  articulated  by 
collaborating  federal  agencies  and  by  participants  attending  division-sponsored  forums  and 
workshops    User  technology'  integration  requirements  and  technical  barriers  to  computer  system 
interoperability  are  compiled  into  a  program  of  work    The  highest  priority  is  given  to  solutions 
that  integrate  various  disparate  technologies  at  a  system  interface  (human,  data,  communication, 
and  application). 

Assessment  of  Priorities  and  the  Priority-Setting  Process 

The  division  has  made  significant  progress  in  the  area  of  articulation  of  priorities. 
Discussion  with  the  division  staff' makes  clear  that  projects  are  selected  in  accord  with  the  needs 
of  industry  and  the  mission  of  the  division.  The  process  by  which  these  priorities  are  set  is  still 
less  visible  and  explicit  than  might  be  desired,  and  the  staff  are  encouraged  to  continue  to  work  on 
the  process  of  making  this  priority  setting  more  formal. 

The  division's  mission  is  appropriately  broad  given  the  rapidity  of  change  in  this  area.  The 
division  is  made  up  of  three  groups:  Software  Engineering,  Distributed  Systems  Engineering,  and 
Office  Systems  Engineering.  Although  software  engineering  is  an  established  area  of  study, 
distributed  systems  engineering  is  less  well  defined,  and  office  systems  engineering  is  a  new  field. 
Thus,  the  groups  are  working  at  differing  levels  of  definition  and  clarity.  The  approach  of 
exploring  new  technologies  as  they  emerge  seems  to  be  working  well.  In  particular,  the  work  in 
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object  technology,  portable  documents,  web  documents,  and  health  care  infrastmcture  is 
consistent  with  the  mission. 

The  division  is  working  with  the  Information  Systems  Engineering  Division  to  begin  the 
process  of  planning  for  the  formation  of  the  Information  Technology  Laboratory.  The  two 
divisions  have  developed  specific  plans  for  how  they  might  contribute  to  both  application 
engineering  and  interoperability  within  the  structure  of  the  new  laboratory.  The  current  state  of 
the  planning  process  is  appropriate,  and  the  staff  are  positively  engaged  in  the  redefinition  of  their 
division's  mission.  Division  staff  exhibit  a  refreshingly  positive  approach  to  the  challenges  and 
opportunities  posed  by  the  creation  of  the  new  laboratory. 


Assessment  of  Technical  Programs 

The  quality  and  effectiveness  of  the  technical  programs  pursued  within  the  division  are 
strong  but  could  possibly  be  improved  fijrther  by  focusing  on  fewer  projects  with  larger 
collaborative  groups. 

With  regard  to  the  status  of  the  equipment  and  facilities,  a  high-technology  area  has  a 
continual  need  for  upgrading  and  enhancing  the  computer  environment,  local  workstations,  and 
communications  and  a  need  for  more  powerful  shared  facilities    In  particular,  the  fijture 
development  of  the  Virtual  Reality  Testbed  will  require  more  investment  and  should  be  carefully 
coordinated  with  divisions  doing  related  work,  e.g.,  the  Visualization  Group  in  the  Computing 
and  Applied  Mathematics  Laboratory  and  the  Advanced  Systems  Division. 


Assessment  of  the  Impact  on  Industry 

Many  projects  have  developed  effective  connections  with  the  standards  community  and 
the  other  laboratory  groups    The  pace  of  development  in  both  the  health  care  and  object 
technology  fields  is  rapid;  these  areas  require  more  resources  and  emphasis  if  the  division  is  to 
make  a  significant  contribution.  The  virtual  reality  field  is  quite  fragmented,  and  division  activities 
directed  toward  establishing  standards  and  protocols  should  yield  immediate  benefit. 

The  impact  on  industry  of  division  programs  is  difficult  to  measure.  The  priorities  being 
set  for  the  National  Information  Infrastructure  initiative  may  represent  one  framework  within 
which  division  activities  might  be  assessed  and  tracked.  The  panel  was  impressed  with  the 
specific  anecdotal  evidence  provided  by  the  staff  regarding  the  industrial  impact  of  division 
programs. 

The  Health  Care  initiative,  the  High-Integrity  Software  System  Assurance  (HISS A) 
project,  the  Integrated  Software  Engineering  Environment  project,  and  research  on  virtual  reality 
development  are  important,  high-visibility  projects    Further,  the  ability  to  effectively  influence 
technology  development  in  these  areas  will  vary  broadly  with  health  care  being  the  most 
problematic.  Health  care  has  a  broad  collection  of  different  organizations,  including  medical 
societies  and  other  federal  agencies,  that  control  and  issue  requirements,  specifications,  and 
standards.  Currently,  MIST  has  no  presence  in  the  medical  community  and  may  have  difficulty  in 
promoting  its  viewpoint. 
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Recommendation 

The  following  is  the  panel's  recommendation  for  the  Systems  and  Software  Technology 
Division. 

The  Systems  and  Software  Technology  Division  should  find  a  way  to  establish  credibility  within 
the  medical  community  for  its  contributions  to  the  Health  Care  initiative. 


Systems  and  Software  Technology  Division  Responses  to  Fiscal  Year  1994 
Recommendations 

Below  are  the  panel's  1994  recommendations  for  the  Systems  and  Software  Technology 
Division,  quoted  fi^om  the  fiscal  year  1994  report,  and  the  division's  response. 

1994  Panel  Recommendation    "The  Systems  and  Software  Technology  Division  should  revisit  its 
objectives  and  strategy  statements  and  revise  them  to  define  more  focused  and  achievable  goals" 
(p.  212). 

Division  Response:  The  division  is  doing  this  in  the  context  of  the  goals  and  objectives 
established  for  the  Information  Technology  Laboratory. 

1994  Panel  Recommendation:  "The  division  should  examine  development  processes  and  develop 
specifications  for  safety-critical  software  development  processes"  (p.  212). 

Division  Response:  The  division  is  following  this  advice  in  the  HISSA  project  and  has  developed 
a  framework  for  the  development  and  assurance  of  high-integrity  software  describing  SRMs. 

1994  Panel  Recommendation:  "The  division  should  give  attention  to  standards  for  process 

engineering"  (p.  212). 

Division  Response:  The  division  is  heavily  involved  in  this  activity. 

1994  Panel  Recommendation:  "The  Virtual  Library  project  should  work  with  industry  to 
establish  a  set  of  requirements  and  specifications  to  support  universal  electronic  document 
exchange"  (p.  213). 

Division  Response:  The  division  has  engaged  in  an  active  program  involving  a  series  of 
workshops  and  a  government  blue  ribbon  panel.  Results  will  be  published  shortly. 

1994  Panel  Recommendation:  "The  division  should  take  a  more  active  role  in  the  SEI  [Software 
Engineering  Institute]  capability  maturity  model,  developing  a  program  with  SEI  and  taking  a 
leadership  role  in  mapping  the  CMM  [coordinate  measuring  machine]  to  the  Generic  Life  Cycle 
Model"  (p.  213). 
Division  Response:  The  division  is  following  this  advice  in  the  HISSA  project. 
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Advanced  Systems  Division 


Mission 


The  Advanced  Systems  Division  develops  commercial  enablers  for  advanced  information 
technologies  likely  to  lead  to  large  new  markets  and  facilitate  the  development  of  the  national 
information  infrastructure.  These  enablers  are  cooperatively  agreed-upon  reference  materials, 
tests,  and  instruments  that  provide  measurement  methods  necessary  to  drive  research  prototypes 
into  a  commercial  paradigm  or  demonstrate  the  commercial  methods  of  two  or  more  competing 
research  paradigms. 


Resources 

The  division  staff  includes  91  full-time-equivalent  employees    Funding  for  fiscal  year  1994 
totaled  $9.2  million  consisting  of  $3.3  million  from  STRS,  $4.6  million  from  OA,  $0.7  million 
from  NIST  competence  funds,  and  $0.6  million  from  the  Information  Technology  Services 
initiative. 


The  Priority-Setting  Process 

The  division's  activity  focuses  on  the  convergence  of  computation,  communication,  and 
display  technology.  Within  this  framework,  the  division  has  chosen  to  concentrate  on  the 
development  of  measurement  methods  for  three  key  advanced  technology  areas:  human-machine 
interfaces,  digital  communication  (broad-band  integrated  services  digital  networks  and  wireless), 
and  advanced  technology  platforms  (parallel  processing,  optical  storage,  and  flat  panel  displays). 
The  division  is  working  with  industry,  academic  researchers,  and  users  through  workshops  and 
collaborations  to  direct  its  efforts  toward  the  needs  of  its  customers. 


Assessment  of  Priorities  and  the  Priority-Setting  Processes 

The  division's  process  of  actively  interfacing  with  industry,  using  high  levels  of  other 
agency  funding  and  informal  methods  to  find  the  leading  technological  edge,  is  adequate  in  the 
absence  of  a  driving  vision  from  upper  management. 

The  division's  activities  in  human-machine  interfacing,  advanced  component  technology, 
and  high-capacity  networking  represent  a  reasonable  view  of  where  the  information  processing 
industry  is  going.  Similarly,  the  division's  interaction  with  industry  through  its  generation  of 
corpora  and  other  test  materials  is  helpful.  The  division's  efforts  in  the  development  of 
holographic  memory  and  three-dimensional  display  are  well  warranted.  Understandably,  the 
selection  of  specific  projects  depends  on  client  interest.  However,  this  tendency  must  be 
tempered  with  a  strong  commitment  to  strategic  objectives. 
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Assessment  of  Technical  Programs 

The  overall  quality  of  the  division's  programs  is  very  high,  but  there  is  still  some  room  for 
improvement.  The  laboratory's  leadership  is  dynamic,  and  overall  researcher  morale  appears  to 
be  good;  there  is  considerable  enthusiasm  for  existing  projects,  and  staff  members  appear  satisfied 
that  they  are  working  at  the  leading  edge  of  technological  development.  The  entrepreneurial 
organization  of  the  division  leads  to  a  strong,  results-oriented  program.  Milestones  are  taken 
seriously,  and  continued  high  levels  of  sponsor  support  suggest  that,  by  and  large,  the  milestones 
are  being  met. 

The  panel  reviewed  the  division  in  terms  of  its  three  program  categories:  human-machine 
interface,  advanced  components,  and  networking.  The  emphasis  on  the  human-machine  interface 
has,  as  its  primary  goal,  the  production  of  corpora  (databases  against  which  various  algorithms 
are  tested).  These  corpora  appear  to  be  well  received,  and  the  division,  in  the  production  and 
analysis  of  these  corpora,  has  played  a  useful  role  in  fostering  synergistic  cooperation  within  the 
wider  research  community.  That  said,  it  is  not  clear  that  the  production  of  corpora  is  as 
technologically  demanding  as  might  be  suggested  by  the  quality  and  quantity  of  personnel. 
Understandably,  their  production  is  necessary,  and  to  be  a  credible  member  of  the  community  in 
which  the  corpora  are  circulated  requires  considerable  technological  depth.  It  should  be  possible 
to  devote  some  of  that  technical  capability  to  other  research  projects  even  if  they  are  not 
metrology  outputs  per  se. 

The  advanced  components  projects  are  technically  excellent  but  require  greater  attention 
to  relevance  and  timely  delivery.  For  example,  the  research  on  flat  panel  displays  and  CD-ROMs 
has  been  valuable,  but  both  programs  should  already  have  been  packaged  and  delivered  to 
industry.  Transfer  of  these  projects  to  industry  would  permit  the  devotion  of  greater  resources  to 
three-dimensional  storage  and  display  technologies    Parallel  processing  work  seems  to  suffer 
from  a  lack  of  relevance:  the  MultiKron  project  was  a  technical  success  but  has  not  yet  been 
successfully  transferred  to  industry,  and  the  S-check  approach  taken  by  the  parallel  computing 
program  to  metrology  is  not  compatible  with  the  approach  industry  has  taken. 

The  division  has  a  contribution  to  make  in  developing  metrology  for  parallel  systems;  it 
seems  to  have  shied  away  from  this  task  because  of  past  problems  over  export  controls  and 
current  controversies  over  possible  bias  among  vendors.  Yet  the  federal  government,  as  the 
mainstay  of  the  parallel  processing  industry,  has  a  strong  and  continuing  need  for  such  metrology, 
and  the  division  ought  to  fmd  a  way  to  fill  this  need.  The  division  maintains  a  node  on  the 
Beltway  Bitway  and  has  played  a  leading  role  in  developing  simulators  for  testing  ATM 
congestion  control  schemes.  The  division  should  more  actively  define  how  it  will  use  its  Bitway 
node  (inter-NlST  applications  are  insufficient  justification  for  such  activity). 

With  regard  to  the  status  of  equipment  and  facilities,  the  panel  perceived  no  evidence  of 
any  inadequacies. 

With  regard  to  data  programs,  the  division  has  achieved  merited  industry  respect  in  its 
development  of  numerous  corpora  covering  test  areas  in  speech  recognition,  handwriting 
recognition,  natural  language  processing,  and  facial  recognition,  among  others.  Standards  per  se 
are  not  a  focus  of  division  effort,  but  the  division  has  nonetheless  made  contributions  in  several 
areas,  particulariy  in  networking. 

In  view  of  the  division's  manifest  technological  strengths,  the  panel  believes  that  the 
division  could  do  much  more,  especially  in  image  and  other  compression,  in  fijndamental  research 
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affecting  the  development  of  a  consistent  semantic  substnicture  for  Nil  communications,  and  in 
problems  that  may  detract  from  the  scalability  of  large  networks. 


Assessment  or  Impact  on  Industry 

Because  one-half  of  the  division's  budget  comes  from  other  agencies,  a  large  share  of  its 
activity  flirthers  the  mission  of  other  parts  of  the  government  (e.g.,  face  recognition  for  the 
Federal  Bureau  of  Investigation  and  handwriting  recognition  for  the  Internal  Revenue  Service). 
The  division  has  several  methods  of  assessing  the  impact  of  its  work  on  industry.  One  method  is 
its  success  in  the  standards  arena:  the  division  participates  in  the  ATM  Forum  and  in  other  areas 
(e.g.,  in  library  retrieval  systems).  Another  method  is  the  degree  to  which  it  has  been  able  to 
attract  industry  to  its  symposia  and  other  forums  or  attract  participants  to  its  CRADA.  A  third  is 
the  degree  to  which  its  published  outputs  (e.g.,  printed  material,  corpora  on  CD-ROM)  are 
requested  by  others. 

The  division's  impact  on  industry  is  growing,  but  it  remains  modest,  particularly  given  an 
annual  budget  of  over  $9  million.  Many  of  its  programs  are  driven  by  researcher  interests  rather 
than  industry  relevance.  The  division's  methods  for  determining  how  well  it  is  doing,  however, 
are  more  than  adequate. 


Recommendations 

The  panel  recognizes  that  the  formation  of  the  Information  Technology  Laboratory  and 
the  appointment  of  a  new  director  will  lead  to  new  thinking  about  the  overall  role  and  mission  of 
what  is  now  the  Computer  Systems  Laboratory  and  its  constituent  divisions.  Although  the  precise 
nature  of  the  new  guidance  cannot  be  known  with  certainty,  it  is  likely  that  it  will  accentuate  the 
role  that  NIST  will  play  in  enabling  the  Nil,  notably  through  advances  in  interoperability,  the 
human-machine  interface,  and  scalability.  The  panel's  recommendations,  accordingly,  are  as 
follows; 

The  Advanced  Systems  Division  should  continue  to  devote  an  increasing  share  of  its  resources  to 
fostering  interoperability,  particularly  in  the  area  of  human-computer  interfaces,  video  and  other 
forms  of  compression  (e.g.,  in  support  of  wireless  communications),  and  complex  networks.  The 
Advanced  Systems  Division,  together  with  other  divisions,  should  begin  work  on  developing 
semantic  substructures  for  national  information  infrastructure  communications,  particularly  at  the 
middleware  layer. 

The  division  should  convert  the  technical  work  it  does  as  a  prerequisite  to  generating  corpora  into 
other  research  products  with  discrete  deliverables. 

The  division  should  find  a  sponsor  for  whom  it  can  develop  benchmark  algorithms  for  parallel 
processing  machines;  this  sponsor  should  be  expected  to  take  ownership  of  these  benchmark 
algorithms. 
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The  division  should  continue  to  pursue  outside  funding.  However,  funding  alone  is  not  evidence 
of  relevance  to  the  NIST  mission.  Project  selection  must  pay  more  attention  to  the  overall  NIST 
mission  and  to  the  mission  of  the  Information  Technology  Laboratory.   Similarly,  it  is  good  when 
investigators'  interests  can  be  accommodated  within  the  confines  of  the  mission  and  vision,  but 
investigator  interests  should  be  a  second-order  effect. 


Advanced  Systems  Division  Responses  ito  Fiscal  Year  1994  Recommendations 

Below  are  the  panel's  1994  recommendations  for  the  Advanced  Systems  Division,  quoted 
from  the  fiscal  year  1994  report,  followed  by  the  division's  response. 

1994  Panel  Recommendation:  "The  [division]  needs  a  baseline  architecture  or  comprehensive 
model  on  which  to  relate  the  various  division  projects.  .  .  ."  (p.  223). 

Division  Response:  The  division  has  concentrated  its  efforts  under  the  three  general  headings  of 
human-machine  interfaces,  digital  communications  technology,  and  advanced  technology 
platforms,  focusing  on  instrumentation,  evaluation,  test  corpora,  and  interoperability  in  each  area. 
This  provides  a  baseline  and  some  projection  into  the  future  for  planning  purposes. 

1994  Panel  Recommendation:  ".  .  .  [More]  work  .  .  .  should  be  oriented  toward  interoperability 
issues  and  toward  developing  standards  derived  from  [those  issues]  .  .  ."  (p.  224). 
Division  Response:  The  division's  work  on  flat  panel  displays,  optical  computing,  Z39.50, 
network  protocols,  and  future  digital  video  deals  with  interoperability  and  portability  issues. 

1994  Panel  Recommendation:  "The  division  should  carefully  plan  to  minimize  disruption  by  the 

planned  move  to  off-campus  space"  (p.  224). 

Division  Response:  The  Advanced  Systems  Division  will  remain  on  the  main  NIST  campus  in 

Gaithersburg. 

1994  Panel  Recommendation:  "The  division  should  consider  development  of  computer 

languages  for  controlling  new  machine  architectures"  (p.  224). 

Division  Response:  The  opportunity  to  pursue  this  avenue  has  not  yet  arisen. 

1994  Panel  Recommendation:  "The  division  should  consider  more  beginning-to-end  testing 
processes  based  on  user  scenarios  to  avoid  oversights  resulting  from  testing  a  technology  without 
a  user  community  perspective"  (p.  224). 

Division  Response:  The  division  is  beginning  this  type  of  effort  with  the  prototype  natural 
language  interface  for  libraries    The  division  plans  further  work  of  this  type  as  the  opportunity 
arises  as  part  of  the  focus  on  human-machine  interfaces. 

1994  Panel  Recommendation:  "Projects  are  dropped  when  they  are  deemed  ready  for 
maintenance.  .  .  .  The  [division's]  transition  process  should  be  better  defined  and  documented"  (p. 
224). 
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Division  Response:  The  division  effects  a  transition  to  new  projects  when  the  technologies  in  an 

existing  project  have  been  accepted  in  the  marketplace  and  no  longer  benefit  from  NIST 

resources. 

J 995  Panel  Comment:  The  panel  recognizes  the  limitations  of  fixed  objective  criteria  for  project 

transition,  but  it  urges  that  transitions  in  some  cases  (e.g.,  MultiKron,  and  CD-ROM  archiving 

metrology)  be  accelerated. 

J  994  Panel  Recommendation:  "The  [division]  should  consider  additional  work  in  several 
interoperability  areas  ...  (p.  224). 

Division  Response:  The  division  is  planning  work  in  the  digital  video  area  by  establishing  a 
testbed  for  investigating  interoperability  issues. 


Information  Systems  Engineering  Division 

Mission 

The  division  conducts  research,  development,  and  technology  transfer  to  industry.  The 
division  delivers  products  and  services  such  as  profiles  and  guidelines,  prototypes  and  reference 
implementations,  tests  and  demonstrations  of  feasibility  and  interoperability,  and  conformance 
tests  and  certification  that  software  products  conform  to  standards.  The  division  provides  direct 
assistance  to  industry  and  local,  state,  and  federal  government  agencies. 

Resources 

Division  staff  include  39  full-time  employees  and  4  part-time  employees.  Funding  for 
fiscal  year  1994  totaled  $5.1  million,  including  $2  2  million  from  STRS,  $2.7  million  from  OA, 
and  $0  2  million  from  NIST  competence  funds. 

The  Priority-Setting  Process 

The  division  obtains  direction  and  support  from  industry  (primarily  information  technology 
developers,  integrators,  and  users),  academia,  and  government  through  workshops,  demonstration 
projects,  and  cooperative  agreements    Progress  is  measured  in  terms  of  products  and  services 
delivered  and  the  degree  to  which  those  products  and  services  are  accepted  in  the  marketplace. 

Assessment  of  Priorities  and  the  Prioritj-Setting  Process 

The  priorities  of  the  division  appear  to  be  appropriate  and  well  balanced.  However,  an 
explicit  process  for  establishing  priorities  is  not  apparent.  The  division  focuses  on  the 
development  of  enabling  technologies  and  is  closely  connected  with  its  customers    Planning  for 
new  activities  tends  to  be  opportunistic  rather  than  strategic. 
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Assessment  of  Technical  Programs 

The  quality  and  effectiveness  of  the  division's  technical  programs  are  very  high.  However, 
the  division  may  be  holding  onto  programs  too  long.  Some  current  projects  could  logically  be 
turned  over  to  industry.  In  terms  of  generating  new  ideas  to  lead  the  industry,  the  division  hosts 
visiting  scientists  and  works  with  industrial  researchers,  on  both  a  national  and  international  level. 
However,  many  of  the  staff  are  fi^om  traditional  data  management  backgrounds,  and  a  systematic 
retraining  and  education  program  should  be  established  to  ensure  that  the  division  maintains  a 
leadership  role  in  changing  the  data  management  environment. 

Facilities  required  to  support  the  development  of  virtual  reality  technology  appear 
to  be  limited.  Video  conferencing  facilities  need  to  be  enhanced  and  may  require  additional  space. 


Assessment  of  Impact  on  Industry 

Projects  pursued  within  the  division  have  generated  a  great  deal  of  interest  on  the  part  of 
industry.  The  division's  Center  for  Applied  Information  Technology  serves  as  a  focal  point  for  its 
interactions  with  industrial  users    Division  support  of  the  database  language  SQL  and 
development  of  test  suites  are  of  immediate  benefit  to  industry. 


Recommendations 

The  following  are  the  panel's  recommendations  for  the  Information  Systems  Engineering 
Division 

The  Information  Systems  Engineering  Division  should  hasten  the  process  by  which  projects  are 
handed  off  to  the  user  community.  .  . 

The  division  should  ensure  that  current  staff  engaged  in  research  related  to  data  management 
technology  maintain  state-of-the-art  expertise 

The  division  should  consider  making  improvements  in  video  conferencing  and  virtual  reality 
facilities  as  resources  allow. 


Information  Systems  Engineering  Division  Responses  to  Fiscal  Year  1994 
Recommendations 

Below  are  the  panel's  1994  recommendations  for  the  Information  Systems  Engineering 
Division,  quoted  from  the  fiscal  year  1994  report,  followed  by  the  division's  response. 

1994  Panel  Recommendaticn:  "The  [division]  needs  to  find  ways  to  off-load  conformance 
testing  to  outside  sources"  (p.  207) 
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Division  Response:  Outside  sources  are  being  used  for  developing  conformance  tests,  for 
conducting  testing,  and  for  r.dministering  the  testing  programs. 

J 994  Panel  Recommendation:  "The  Graphic  Software  Group  .  .  .  should  expand  its  scope  to 
include  PC  workstation  graphical  user  interfaces"  (p.  207). 
Division  Response:  The  group's  scope  already  includes  this  activity. 

1994  Panel  Recommendation:  "The  [division]  must  clarify  and  better  coordinate  its  joint 
program  on  virtual  reality  with  the  Systems  and  Software  Technology  Division"  (p.  207). 
Division  Response:  The  division  is  working  with  the  Systems  and  Software  Technology  Division 
on  a  joint  project.  Systems  Integration  for  Manufacturing  Applications,  utilizing  virtual  reality 
technology. 

1994  Panel  Recommendation:  "The  division  must  plan  more  explicitly,  including  planning  for 
new  technology  areas.  It  seems  to  emphasize  support  for  its  ongoing  activities  rather  than 
support  for  new  research  activities"  (p  208). 

Division  Response:  The  division  engages  in  new  activities  when  those  activities  are  requested  by 
industry.  Such  requests  are  often  in  the  context  of  ongoing  activities,  which  biases  the  division's 
projects  in  that  direction. 

J994  Panel  Recommendation:  "The  division  should  take  an  active  role  in  software  reuse 
technologies"  (p.  208). 

Division  Response:  The  division  is  interested  in  assuming  an  active  role,  but  a  lack  of  fijnding  has 
limited  the  division's  efforts 

J994  Panel  Recommendation:  "The  division  should  study  automatic  testing  methods  for 
software  standards"  (p.  208). 

Division  Response:  Automatic  testing  methods  are  used  whenever  possible,  but  there  are 
limitations  imposed  by  the  nature  of  the  test  suites  available  or  by  the  nature  of  the  testing  itself 
The  division  is  currently  studying  two  automatic  testing  methods  that  automatically  generate  code 
from  detailed  specifications.  The  division  plans  to  use  one  of  these  methods  to  develop 
conformance  tests  for  a  new  image  processing  standard 

J994  Panel  Recommendation:  "The  division  should  use  the  Computer  Security  Division's 

strategic  plan  as  a  model  for  its  own  planning"  (p.  208). 

Division  Response:  The  division  has  adopted  the  recommendation. 

J994  Panel  Recommendation:  "The  division  should  revisit  the  panel's  fiscal  year  1993 

recommendation  on  conformance  tests"  (p.  208). 

Division  Response:  This  issue  has  been  revisited  and  will  continue  to  receive  attention. 
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Mrs.  MORELLA.  Thank  you,  Dr.  Roskies. 
Dr.  Brownstein? 

STATEMENT  OF  DR.  CHARLES  N.  BROWNSTEIN 

Dr.  Brownstein.  Mrs.  Chairwoman,  Ms.  McCarthy,  thank  you 
for  the  opportunity  to  testify  today.  I  am  the  Executive  Director  of 
the  Cross-Industry  Working  Team  which  represents  45  of  our  lead- 
ing information  technology  and  service  companies.  I  will  leave  you 
with  some  of  our  public  materials  about  the  future  of  our  informa- 
tion infrastructure  for  your  Committee's  use. 

I  bring  some  additional  perspective  as  a  member  of  the  Public 
Policy  Committee  of  the  ACM  and  from  time  spent  at  the  National 
Science  Foundation,  where,  among  other  things,  I  managed  the  Di- 
rectorate for  Computer  and  Information  Science  for  a  while  and 
worked  fairly  closely  with  NIST. 

I  am  most  pleased  to  be  here,  however,  as  a  resident  of  Mary- 
land's 8th  Congressional  district.  I'd  like  to  thank  you  for  getting 
our  lights  turned  on. 

[Laughter.] 

Dr.  Brownstein.  I  provided  written  testimony  for  the  record, 
and  so  I'd  just  like  to  make  a  few  comments  to  highlight  what  our 
members  told  me  when  I  asked  them  about  the  role  and  utility  of 
NIST's  information  technology  activities. 

I  think  I  would  associate,  first,  myself  and  the  XIWT  with  the 
opening  statements.  Few  issues  affect  our  economic  health  as  those 
which  affect  our  information  industries  and  the  goods  and  services 
which  they  enable.  Given  the  critical  place  of  Federal  investments 
in  the  evolution  of  this  industry,  your  Committee's  understanding 
and  actions  will  affect  every  American  profoundly. 

It's  well  documented  that  what  are  popularly  misunderstood  as 
revolutions  in  this  field  are  the  product  of  knowledge  and  human 
resources  generated  by  the  longstanding  partnership  of  the  govern- 
ment, industry,  and  the  academic  community.  Fundamental  tech- 
nology and  applications  research  is  the  cradle  of  much  of  the  visible 
entrepreneurial  activity  that  brings  products  to  market. 

It's  less  well  documented  and  scarcely  understood  outside  of  in- 
dustry that  the  efficient  extension  of  these  technologies  to  mass  de- 
ployment is  dependent  on  a  reasoned  degree  of  consensus,  agree- 
ment, and  common  ground,  and  that's  why  industry  values  NIST 
as  a  competent,  neutral  asset  in  the  competitive  exchange  of  infor- 
mation technology  arena. 

XIWT  member  companies  believe  that  the  Information  Tech- 
nology Laboratory  has  made  important  contributions  by  its  support 
of  industry's  voluntary  standards  development  processes  and  in  fa- 
cilitating some  particularly  difficult  areas  of  fundamental  research, 
and  that  it  should  continue  to  do  so  in  the  future.  In  particular, 
XIWT  member  companies  believe  that  NIST  should  promote  mar- 
ketplace solutions  to  the  selection  of  standards.  We  believe  the  de- 
sired course  of  government  is  to  avoid  intervention  through  the 
promotion  or  legislation  of  particular  technical  standards,  however. 

ITL  should  also  support  the  voluntary  industry  standards  proc- 
ess by  offering  its  expertise  in  metrology  to  assist  the  understand- 
ing of  complex  alternative.  Industry  needs  the  ability  to  select  tech- 
nical standards  by  its  own  processes.  We  are  in  an  era  of  continu- 
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ous  improvement  in  the  underlying  technologies  for  information 
processing  and  communications.  An  abundance  of  new  and  compet- 
ing solutions  to  information  processing  and  communications  re- 
quirements have  resulted  from  our  public  investment  in  research 
and  education  over  the  past  several  generations. 

Government  has  a  role  in  facilitating  the  process  of  achieving  in- 
dustry consensus  by  providing  vendor  neutral  methods  for  perform- 
ance assessment,  the  independent  trusted  data  about  performance, 
and  neutral  forums  for  the  consideration  of  contending  issues.  That 
is  not  to  say  that  NIST  should  create  or  support  such  forums.  In- 
dustry has  the  resources  and  incentives  to  do  that. 

But  where  invited,  NIST  can  provide  a  useful  neutral  participant 
role,  particularly  early  in  the  process  of  considering  standards,  sub- 
sequently where  there  is  a  clear  industry  consensus  about  the  need 
and  a  willingness  to  collectively  provide  direct  support  for  such  ac- 
tivities. There  is  a  balance  that  has  to  be  achieved  about  competing 
priorities  and  it  is  incumbent  on  NIST  to  work  with  industry  to  as- 
sess priorities  but  then  to  make  the  hard  choices  that  competing 
interests  would  always  generate. 

Third,  NIST  should  contribute  to  the  U.S.  Government  position 
in  international  standards  activities,  helping  government  share 
this  burden  with  industry,  since  the  management  of  standards  has 
strategic  trade  implications.  Industry  may  not  easily  aggregate  its 
own  interests  and  our  European  and  Asian  trading  partners  strate- 
gically fund  both  government  and  industry  involvement. 

Finally,  NIST  should  participate  with  other  agencies  in  the  sup- 
port of  fundamental  research,  using  its  special  expertise  in  stand- 
ardized testing  and  performance  evaluation,  as  appropriate.  Gov- 
ernment has  a  very  important  role  to  play  in  the  generation  of  new 
fundamental  knowledge  that  all  of  society  can  exploit.  Most  of  this 
responsibility  and  the  bulk  of  public  resources  are  in  other  agen- 
cies, such  as  the  National  Science  Foundation  and  DARPA,  which 
together  support  60  to  75  percent  of  all  academically-based  fun- 
damental research  and  information  technology. 

These  agencies  specialize  in  forefront  risky  cutting-edge  research 
and  development  in  technology  and  applications  and  usually  have 
little  patience  to  follow  up  with  some  of  the  more  critical  work  that 
needs  to  be  done  to  support  even  research  activities  over  the  long 
run.  NIST's  Information  Technology  Laboratory  thus  compliments 
these  agencies  in  important  and  difficult  technical  areas  in  which 
standard  test  suites  and  performance  evaluation  methods  and  cri- 
teria facilitate  technology  improvements  that  all  of  industry  can  ex- 
ploit. The  laboratory's  intramural  research  and  open  scholarly  pub- 
lication in  selected  areas  is  also  essential  to  maintaining  its  own 
staff  capability  and  credibility  in  this  area. 

Thank  you  very  much.  I'll  be  pleased  to  respond  to  any  questions. 

[The  prepared  statement  of  Dr.  Brownstein  follows:] 
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Mrs.  Chairman  and  members  of  the  Subcommittee,  thank  you  for  the  opportunity  to  testify 
on  technical  and  policy  issues  facing  the  information  industry  and  the  role  of  the 
Information  Technology  Laboratory  of  the  National  Institutes  of  Standards  and  Technology 
in  dealing  with  these  issues.  My  name  is  Charles  Brownstein.  I  am  the  Executive  Director 
of  the  Cross-Industry  Working  Team  (XIWT),  a  coalition  of  45  leading  companies  in  the 
information  services,  communications  and  computing  industry,  which  are  working 
together  to  accelerate  the  development  of  the  national  information  infrastructure.  This 
position  brings  me  into  continual  contact  with  the  technical,  business  and  public  policy 
issues  affecting  many  of  our  leading,  most  innovative  information  technology  companies. 

I  bring  additional  perspective  as  a  member  of  the  Public  Policy  committee  of  the 
Association  for  Computing  Machinery  (ACM),  and  from  my  background  as  a  Trustee  of 
the  Internet  Society,  the  capstone  organization  for  Internet  standards;  and  from  service  at 
the  National  Science  Foundation,  where,  in  various  capacities  from  1985  to  1991, 1 
managed  the  Directorate  for  Computer  and  Infomiation  Science  and  Engineering,  and 
chaired  or  participated  in  many  interagency  committees  and  cross-sector  activities  which 
involved  the  people  and  capabilities  of  the  laboratories  which  are  the  subject  of  these 
hearings. 

Background 

America's  information  industries  and  the  progress  they  make  in  deploying  the  broad  set  of 
information  technologies  that  fit  under  their  rubric  of  "national  information  infrastructure," 
have  had,  and  will  continue  to  have,  substantial  leverage  on  every  aspect  of  our  economic 
and  social  well  being,  national  competitiveness  and  security.  Commerce  of  every  sort, 
government  performance,  health  care,  education,  publishing,  scientific  and  engineering 
research,  entertainment,  national  defense,  law  enforcement,  politics  and  our  personal 
communications  capabilities  are  all  profoundly  affected  today  by  advances  in  iriformation 
technologies  and  the  applications  they  enable. 

In  the  practical  near  term  and  over  the  long  run,  few  issues  affect  the  future  economic 
health  facing  the  nation  as  those  affecting  our  information  industries  and  the  cascade  of 
goods  and  services  which  they  enable.  With  the  recent  changes  in  the  laws,  the  astringent 
forces  of  competition,  and  the  likely  future  decline  in  knowledge  generating  capacity  due  to 
financial  pressures  on  both  government  and  industry,  the  Congress'  comprehension  of  the 
issues  we  discuss  today  will  affect  every  American  profoundly  in  the  future. 

American  industry  is  noted  for  information  technology  "revolutions"  such  as  converge 
today  in  the  Internet —  computers,  telecommunications,  and  user  friendly  multimedia 
applications.  It  is  very  well  documented  that  these  revolutions  are  a  product  of  the 
knowledge  and  human  resources  generated  by  the  long  standing  partnership  of  the  federal 
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government,  industry  and  the  academic  comity  in  fundamental  technology  and  applications 
research,  and  the  subsequent  entrepreneurial  activity  of  our  economic  system.  It  is  less  well 
documented  and  scarcely  understood  outside  of  industry,  that  the  efficient  extension  of 
these  technologies  to  mass  deployment,  where  they  are  brought  to  bear  on  everyone's 
lives,  is  dependent  on  a  reasoned  degree  of  consensus,  agreement,  and  common  ground. 

All  of  which,  perhaps,  explains  why  industry  values  the  presence  of  NIST  as  a  competent 
neutral  party,  and  views  it  as  a  national  asset  in  the  competitive,  changeable  information 
technology  arena.  XIWT  member  companies  believe  that  NIST's  Information  Technology 
Laboratory  has  made  important  contributions  to  the  advance  of  technology  and  the  health 
of  our  industry  by  its  efforts  to  support  industry's  standards  development  process  and  in 
facilitating  the  conduct  of  some  particularly  difficult  areas  of  fundamental  research,  and 
should  continue  to  do  so  in  the  future. 

Issues  Facing  America's  Information  Technology  Industry 
There  are  many  kinds  of  issues  facing  our  information  industries.  Some  involve 
technology  in  various  application  domains-  such  as  commerce,  health  care,  entertainment 
content;  others  involve  enabling  technologies  and  services,  such  as  security  or  intellectual 
property  rights  management;  still  others  involve  the  interoperability  of  particular  systems, 
such  as  applications  software  or  telecommunications  services.  Continual  technology  and 
marketplace  evolution  makes  these  continual  issues,  best  dealt  with  by  technical  experts. 
History  tells  us  that  they  can  be  readily  dealt  with  on  a  continuing  basis,  (/some  larger 
issues  involving  the  interplay  of  public  policy  and  technology  are  understood.  Thus  my 
testimony  is  focused  on  four  key  issues. 

First  and  foremost,  is  the  issue  of  government  intervention  in  the  marketplace.  The 
industry  needs  the  freedom  to  compete.  Recent  changes  in  the  laws  are  designed  to 
promote  competition  in  order  to  drive  down  prices  and  drive  up  innovation  by  providing 
more  choice  to  consumers.  The  changes  can  succeed  to  meet  these  goals  if  industry  is  able 
to  operate  in  open  free  markets,  to  offer  innovative  products  in  every  domain  of  interest  to 
consumers,  and  to  permit  consumers  to  determine  which  products  and  services  have  value. 
Government  has  a  legitimate  role  in  ensuring  that  markets  are  open,  and  in  being  vigilant 
against  actions  of  the  public  or  private  sector,  which  foreclose  the  ability  to  offer  service 
and  product  innovations  to  consumers.  We  thus  believe  that  the  desired  course  of 
government  is  to  avoid  intervention,  particularly  to  avoid  the  promotion  or  legislation  of 
particular  technical  standards. 

Second,  industry  needs  the  ability  to  select  technical  standards  by  its  own  processes.  We 
are  in  an  era  of  continuous  improvement  in  the  underlying  technologies  for  information 
processing  and  communications.  We  are  reaping  the  seeds  sown  in  our  public  investment 
in  research  and  education  over  the  past  several  generations,  in  the  form  of  an  abundance  of 
new  and  competing  solutions  to  information  processing  and  communications 
requirements.  To  employ  these  solutions  most  effectively  and  efficiently,  industry  needs 
the  ability  to  work  within  the  competitive  environment,  to  achieve  the  degree  of  consensus 
that  permits  widespread  deployment  of  technology  and  systems.  Government  has  a  role  in 
facilitating  the  processes  of  achieving  industry  consensus  by  providing  vender  neutral 
methods  for  performance  assessment,  independent  trusted  data  about  performance,  neutral 
forums  for  the  consideration  of  contending  issues,  and  by  stimulating  institutions  for 
continual  assessment  and  assurance  of  performance  for  the  marketplace. 
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With  respect  to  both  technology  and  its  application,  such  consensus  may  deal  with 
engineering  and  business  practices,  with  performance  characteristics  of  components  or 
systems,  or  with  specifications  for  the  makeup  of  components  and  operations.  These  can 
be  achieved  most  effectively  where  contending  ideas,  technologies,  people  and  companies 
share  the  common  ground  of  broad  agreement  on  the  architecture —  the  overall  plan  for 
systems  and  the  rules  that  bind  them  together.  Subsidiary  agreements  on  standards  cap 
very  human  processes  of  generating  common  understandings.  There  is,  however,  no 
simple  "right"  process  which  actually  determines  standards  in  a  truly  open,  competitive 
marketplace.  The  lack  of  a  process  may  do  it:  a  dominant  vendor  sometimes  sets  a 
standard,  de  facto,  by  monopoly  power.  Or,  groups  of  vendors  may  simply  jointly  settle 
on  a  particular  standard  to  gain  market  position.  Or,  engineers  may  propose  ideal  solutions 
which  attract  the  attention  of  vendors  to  build  within  a  proposed  framework.  Similarly, 
there  is  no  single  way  to  get  to  the  "best"  standard,  nor  any  guarantee  that  any  particular 
standard  will  be  worth  keeping  in  the  face  of  opportunity  created  by  innovation. 

Nonetheless,  industry  and  consumers  value  standards,  and  in  practice,  industry  the  world 
over  engages  in  voluntary,  industry-led  forums  to  work  together  to  agree  on  fundamentals 
and  on  specifications  which  they  publish  openly  for  participants  to  use  in  the  design  of 
products.  Standards  may  reduce  uncertainty  about  what  to  build  and  sell,  may  assure  that 
parts  and  systems  from  diverse  manufacturers  will  work  together,  may  generate 
economies  of  scale,  allowing  a  large  number  of  a  product  to  be  created  because  of  the 
likelihood  that  there  will  be  market  acceptance  based  on  confidence  that  parts  and  systems 
will  perform  as  advertised.  Standards  may  be  important  to  consumers  as  well.  If  there  are 
multiple  providers  of  interchangeable  devices,  competition  reduces  prices.  If  devices  and 
systems  interoperate  effectively,  the  reach  and  utility  of  networks  expands  for  everyone  as 
the  market  grows. 

One  of  the  keys  to  the  advancement  of  technology  in  the  US,  and  the  success  of  US 
companies  in  other  markets,  has  been  the  wide  acceptance  of  standards  developed  in  open 
forums,  and  in  the  power  of  the  marketplace  rather  than  fiat,  to  weed  out  inferior  products. 
Both  industry  and  the  consumer  make  good  use  of  information  about  performance  in 
making  their  choices.  Generating  confidence  in  technology  performance  requires  defining 
acceptable  metrics  of  performance  and  of  verification  of  facts.  A  neutral  participant,  one 
that  is  independent  of  particular  vendors  participating  in  the  deliberative  process  of 
voluntary  industry  standards  organizations  is  needed  to:  provide  standardized  test 
procedures;  administer  system  benchmark  tests,  certify  that  adequate  testing  has  occurred, 
and  facilitate  the  efficient  resolution  of  engineering  disputes.  Practical  issues  involve  getting 
common  performance  test  parameters  and  measures,  designing  standard  methodologies, 
and  providing  the  benchmarking  data. 

This  is  not  to  say  that  NIST  should  attempt  to  provide  forums:  industry  has  the  resources 
and  the  incentives  to  do  that.  In  some  areas,  NIST  has  been  invited  to  provide  a  critical 
neutral  "host"  role  which  has  allowed  all  segments  of  industry  to  have  a  fair  hearing.  In 
some  areas,  NIST-  assisted  forums  have  facilitated  the  mteraction  of  vendors  and 
consumers  early  in  the  process  of  considering  standards.  The  importance  of  this  neutrality 
will  grow  as  cross-industry  competition  increases  in  the  wake  of  changing 
telecommunications  laws.  Should  NIST  step  down  from  the  roles  it  has  evolved,  other  less 
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"vendor-neutral"  coordinators  may  step  forward,  and  the  benefits  of  open  competition  in 
open  markets  may  be  lost. 

My  third  issue  involves  international  competition.  America's  information  industry  is 
thriving  on  open  global  competition.  It  can  continue  to  do  so  only  with  adequate 
representation  in  international  governmental  deliberations  about  standards.  Foreign 
governments  may  specify  requirements  (e.g.,  conformance,  certification)  concerning  the 
sale  of  products  or  services  within  their  countries  (even  though  such  requirements  often 
add  to  the  cost  of  and  slow  the  development  and  deployment  of  new  information 
infrastructure  products  and  services).  These  governments  may  also  require  that 
negotiations  on  how  to  meet  these  requirements  be  carried  out  between  governments, 
rather  than  by  voluntary  industry  bodies.  In  such  cases,  the  U.S.  needs  a  technically 
competent  governmental  representation  and  a  channel  for  U.S.  industry  to  participate  in 
such  negotiations.  The  Information  Technology  Laboratory  can  play  an  important  role  by 
providing  technical  support  to  both  government  and  industry. 

Fourth  and  finally  is  the  issue  of  sustaining  knowledge  resources  for  the  future.  With 
improved  competition,  industry  is  driven  to  deal  ever  more  intensely  with  "bottom  line" 
considerations  of  consumer  acceptance,  cash  flow,  and  the  pressures  of  profitability,  at  the 
expense  of  long  term  knowledge  and  man  resource  considerations.  Industry  needs  a 
continuing  supply  of  new  knowledge  and  new  human  resources,  which  is  beyond  its 
capacity  or  direct  economic  interest  to  create.  The  government  has  an  important  role  to  play 
in  the  generation  of  new  fundamental  knowledge  which  all  of  society  may  exploit. 

Most  of  this  responsibility,  and  the  bulk  of  public  resources  is  in  agencies  such  as  the 
National  Science  Foundation  and  DARPA,  which  together  support  from  60-  75%  of 
academically  based  fundamental  research  in  information  technologies.  These  agencies  (and 
others  with  nationally  critical  missions  such  as  health,  national  security,  and  space)  invest 
in  forefront,  risky,  cutting  edge  research  and  development  in  technology  and  applications. 
NIST's  Information  Technology  Laboratory  is  an  important  component  of  this  well  proven 
knowledge  generation  system.  It  provides  a  suitable  venue  for  long  term  inter-sector 
collaboration  in  several  interesting  and  difficult  technical  areas  in  which  standard  test  suites 
and  performance  evaluation  methods  and  criteria  facilitate  technology  improvements  that 
all  of  industry  can  exploit.  The  Laboratory's  intramural  research,  and  open  scholarly 
publication  in  selected  areas  is  essential  to  maintaining  capability  in  this  arena.  ^ 

In  summary,  NIST's  Information  Technology  Laboratory  should: 

1)  promote  marketplace  solutions  to  the  selection  of  standards; 

2)  support  the  voluntary  industry  standards  process,  particularly  by  offering  it's  expertise 
in  metrology  to  assist  the  understanding  of  competing  alternatives; 

3)  contribute  to  the  US  position  in  international  standards  activities;  Government  should 
share  this  burden  with  industry  since  the  management  of  standards  has  strategic  trade 
implications.  Industry  may  not  easily  aggregate  its  interests,  and  our  European  and  Asian 
trading  partners  presently  fund  both  government  and  industry  involvement. 

4)  participate  with  other  agencies  of  government  in  the  support  of  fundamental  research, 
using  its  special  expertise  in  standardized  testing  and  performance  evaluation  as 
appropriate. 
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Addendum 
Industry  Comments  on  the  Value  of  the  NIST  Information  Technology  Laboratory 


-Telecommunications 

"As  you  may  already  be  aware,  that  802.14  is  a  key  enabling  residential  broadband 
technology  that  leverages  the  ubiquitous  cable  TV  infrastructure  taking  decades  and  billions 
to  build. 

To  provide  affordable  high  speed  intemet  access  (and  other  multimedia  applications)  for 
consumers  and  small  businesses,  infrastructure  and  networking  products  (such  as  cable 
modems)  must  be  based  on  open  standards.  IEEE  Project  802.14,  has  provided  a  structure 
for  world-wide  participation  to  define  a  best-in-class  access  method  and  transmission 
standard  (MAC/PHY  layers)  so  that  interoperative  products  can  be  based. 

(NIST's)  group  has  participated  and  contributed  to  the  evaluation  of  MAC  (medium 
access  control)  proposals  from  competing  participants.  NIST's  role  as  a  neutral  party  has 
been,  and  will  be  increasingly  important  for  the  group  to  converge  to  a  standard  in  a  timely 
fashion. 

We  would  very  much  like  to  see  (NIST)  continue  work  on  the  Common  Simulation 
Platform  to  facilitate  the  selection  of  protocol  features  based  on  technical  merits  and  to 
ensure  future  interoperativity  of  implementations  based  on  802.14. 

We  need  the  continuity  of  the  work  being  started  (NIST)  to  ensure  a  robust  and  deployable 

standard." 

(Digital  Equipment  Corporation) 

"  Examples  of  Positive  Partnerships  (past/present  actions):  1 )  Support  of  a  single  national 
standards  on-line  network  (NSSN)  -  NIST  positive  and  pursued  activity  even  though 
ANSI  was  (at  first)  hesitant  2)  Support  of  PAS  (publicly  available  specifications)  RS 
(reference  specification)  positions/documents  in  JTC  1,  participate  in  standards  meetings, 
facilitate  dialogs  with  potential  PAS  providers  3)  Participate  and  bring  in  global 
perspectives  of  voluntary  testing  and  certification  to  the  North  American  Interoperability 
Policy  Council;  convey  North  American  policies  to  other  national  government  agencies  4) 
Involved  in  and  supportive  of  US  industry  activities  to  promote  the  GEI  such  as  the 
Information  Infrastructure  Standards  Panel;  led  activity  to  develop  a  common  framework, 
provided  tutorials  to  promote  the  framework,  sponsored  health  care  activity  to  identify  the 
standards  that  will  be  required  for  that  industry." 
(HP) 

"  -  NIST  has  had  active  and  contributing  participation  in  major  information  technology 
standards  development  organizations  such  as  the  Accredited  Standards  Committee  X3  and 
the  associated  JTC  1  TAG.  We  have  observed  the  NIST  members  assume  roles  in  sub- 
committee work  and  international  delegations  to  promote  national  views  on  critical 
technical  and  policy  matters. 
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-  The  Information  Infrastructure  Standards  Panel(IISP),  an  ANSI 

sponsored  organization  chartered  to  identify  requirements  for  standards  that  will  be  needed 
to  develop  the  NII/GII,  has  benefited  from  NIST  participants.  In  particular,  one  NIST 
individual  was  responsible  for  leading  the  development  of  a  three  dimensional  framework 
now  being  used  by  the  organization  as  a  guide  for  this  requirement  identification.  Of  note, 
this  individual  has  also  been  instrumental  in  working  with  subgroups  on  the  use  of  the 
framework  which  has  led  to  numerous  quality  outputs. 

-  In  the  IISP  context,  another  NIST  participant  has  been  heavily 

involved  in  coordinating  the  use  of  the  framework  with  the  national  efforts  on  healthcare 
informatics.  This  activity  is  the  most  complex  to  date  and  the  NIST  individual  has  been 
coordinating  most  effectively  between  the  IISP  and  the  person  who  is  the  Executive 
Director  of  the  Medical  Records  Institute,  ANSI  HISB  (ANSI  Healthcare  Informatics 
Standards  Board). 

In  summary,  Apple's  experiences  with  NIST  participation  in  these  various  venues  has  been 
positive.  The  individuals  we  have  worked  with  have  been  knowledgeable,  cooperative,  and 
effective  contributors.  " 
(Apple  Computer) 

"  At  the  ANSI  Tl  Al  meetings  which  develop  telecommunications  standards  they  have 
been  hosts  to  encourage  industry  gatherings  which  facilitates  the  development  of  standards. 
Standards  are  important  because  they  drive  down  the  cost  for  consumers  since  there  are 
multiple  providers  of  interchangeable  devices.  Manufacturers  benefit  because  it  allows  a 
large  number  of  a  product  to  be  created  because  of  having  a  large  market  acceptance.  In 
Tl  Al  they  performed  perception  tests  to  determine  how  well  people  liked  different  audio 
and  video  algorithms  that  performed  compression. 

I  have  also  worked  with  NIST  in  the  ATMForum  where  they  acted  as  an  unbiased  expert 
to  verify  technical  matters.  When  there  are  multiple  ways  of  solving  a  problem,  it  is  very 
useful  to  have  a  knowledgeable  entity  such  as  NIST  explore  the  impact  and  complexity  that 
proposals  would  have  on  the  outcome  of  the  standard.  This  service  removes  the  perception 
that  an  advocate  would  have  in  evaluating  the  impact  of  their  idea. 

A  third  service  that  1  have  observed  from  NIST  is  in  the  IEEE  802.14  committee  that  is 
developing  Cable  Modem  standards.  I  asked  David  Su  to  attend  because  of  the  need  to 
have  independent  confirmation  of  proposals.  There  are  1 30  people  attending  from  a 
number  of  companies  and  there  have  been  17  MAC  proposals  presented.  NIST  has  the 
background  to  help  devise  tests  that  may  duplicate  the  type  of  data  traffic  in  which  we 
eventually  may  operate  the  modem.  It  is  important  to  stress  the  MAC  proposals  against 
this  envelope  to  create  a  viable  product  for  the  consumer. 

Since  there  are  a  spectrum  of  user  skills  in  the  committee,  not  all  proponents  have  the 
expertise  to  handle  the  nuances  of  simulation  that  NIST  can  perform.  Evaluating  a  number 
of  protocols  using  the  identical  traffic  source  and  measuring  the  same  parameters  allows  a 
fair  comparison  of  the  proposals.  This  will  then  allow  the  committee  to  do  its  job  faster  and 
with  more  certainty  because  of  the  assistance  of  NIST." 
(Scientific  Atlanta) 
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"In  1988/89,  SWBT  in  Technology  Planning  worked  with  the  National  Institute  of 
Standards  ( then  known  as  the  National  Bureau  of  Standards),  as  key  players  in  the 
establishment  of  the  North  American  ISDN  Users'  Forum  (NIUF).  Richard  Stephenson, 
SWBT,  was  the  first  chairman  of  the  ISDN  Implementors  Workshop  which  was  one  of 
two  major  arms  of  the  NIUF  ( the  other  being  the  ISDN  Users'  Workshop).  The  overall 
chairman  of  the  NIUF  was  a  high  level  director  at  NIST  named  Shukri  Wakid.  The  NIUF 
is  still  an  active  organization  in  which  NIST  plays  a  major  part.  SWBT  continues  to  attend 
NIUF  meetings." 
(Southwestern  Bell) 

-Text  Retrieval 

"The  IBM  T.J.  Watson  Research  Center  develops  search  technology  for  applications  which 
deal  with  large  collections  of  multimedia  documents,  such  as  digital  libraries  and  the 
world-wide  web.  Much  of  our  work  focuses  on  text,  since  text  search  is  the  most 
developed  of  all  content  searching  techniques.  Even  applications  which  deal  with  other 
content  types,  such  as  image  and  video,  typically  have  a  text  search  component.  Also,  in 
recent  years,  traditional  databases  have  been  extended  to  manage  unstructured  data  such  as 
text,  and  effective  text  search  is  needed  in  those  applications. 

As  we  experiment  with  new  approaches  to  text  search,  it  is  important  to  measure  how 
effective  they  are.  However,  getting  objective,  quantitative  measurements  of  the 
effectiveness  of  information  retrieval  systems  is  very  difficult,  because  the  measurements 
hinge  on  human  judgments  regarding  the  relevance  of  large  numbers  of  documents.  It  is 
quite  difficult  for  any  single  company,  still  less  a  university,  to  muster  the  resources  needed 
to  do  this.  Therefore,  in  1996,  IBM  plans  to  participate  in  the  TREC  conference  organized 
by  NIST,  and  to  use  it  to  assess  some  text  search  systems  being  developed  within  IBM. 

TREC  provides  two  benefits.  First,  it  is  a  forum  in  which  commercial  text  search  products, 
commercial  prototypes,  and  research  systems  can  be  evaluated  using  standard  protocols. 
The  value  of  this  for  a  company  like  IBM  is  great.  Different  ways  of  providing  value  to 
users  can  be  quantitatively  assessed  and  objectively  compared,  which  is  one  of  the  most 
valuable  ways  to  evaluate  advances  in  the  state  of  the  art.  In  order  to  do  the  comparison 
properly,  all  parties  must  use  the  same  data  and  methodology.  NIST  plays  a  valuable  role 
in  providing  the  benchmarking  data  and  designing  standard  methodologies.  Second,  NIST 
is  a  neutral  party  that  is  independent  of  any  commercial  competitor.  Thus  we  are  assured 
that  the  protocols,  data  and  evaluation  procedures  used  in  TREC  are  not  selected  in  order  to 
favor  any  product  or  system. 

We  believe  that  participation  in  TREC  will  help  our  company  develop  products  which 
bring  improved  functionality  and  quality  to  their  users  by  participation  in  an  open,  neutral 
and  effective  evaluation  process." 
(IBM  Thomas  J.  Watson  Research) 

"  I  am  writing  to  comment  on  the  value  of  NIST  research  in  text  retrieval  evaluation  to 
Bellcore  and  the  economy.  Information  retrieval  is  one  of  the  success  stories  of  US 
industry.  Trade  statistics  for  1994  show  US  exports  of  information  services  at  $3.2B 
against  imports  of  $0.4B;  even  to  Japan  the  US  exported  $400M  of  information  services 
and  imported  only  $10M.  NIST  (under  its  previous  name  of  NBS)  has  been  involved  in 
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research  in  information  retrieval  for  decades  and  has  helped  stimulate  and  develop  this 
industry. 

Most  recently,  NIST  has  been  sponsoring  a  series  of  evaluation  tests  known  as  TREK 
(Text  Retrieval  Evaluation  Confer-  encase).  These  experiments  have  allowed  many 
companies  and  universities  to  compare  current  text  processing  methods  and  improve  our 
algorithms  and  their  implementations.  Bellcore  has  participated  in  these  competitions 
several  times,  and  we  are  proud  of  our  performance,  which  has  demonstrated  the 
flexibility,  power  and  generality  of  our  Latent  Semantic  Indexing  (LSI)  technology. 

The  advantages  of  having  this  competition  for  Bellcore  include  our  ability  to  test  and  polish 
our  algorithm;  to  obtain  public  attention  and  possible  licensees  for  our  patents;  and  to 
observe  other  retrieval  projects  and  see  what  we  can  learn  from  their  technology.  We  have 
been  able  to  extend  applications  of  our  LSI  algorithms  to  areas  beyond  text  retrieval  and  are 
currently  experimenting  with  research  prototypes  in  these  areas,  such  as  entertainment. 

Others  have  similarly  been  able  to  learn  from  our  efforts.  For  example,  the  company  that 
markets  the  Excite  search  system  on  the  Internet  has  publicly  credited  LSI  as  the  origin  of 
their  algorithms  (see  Wired  magazine  for  May  1996).  The  information  retrieval  software 
industry  as  a  whole  has  gone  from  under  $IOOM  of  sales  in  1990  to  about  $650M  in  1996 
(estimated). 

I  feel  that  NIST  has  performed  a  very  valuable  service  to  the  information  retriev£il  business 
community  by  running  the  TREC  project  and  has  helped  keep  the  US  in  the  lead  in  this 
area  of  research  and  stimulated  the  industry.  I  hope  it  will  be  able  to  continue  doing  similar 
projects." 
(Bellcore) 

-Speech  recognition 

It  was  in  1980  that  TI  and  NIST  (then  NBS)  collaborated  to  distribute  standard  speech 
corpora  to  evaluate  various  automatic  speech  recognizers  on  a  common  basis.  Thus 
dawned  a  new  era  in  which  all  of  DARPA's  speech  technology  development  flourished;  it 
did  so,  mainly  because  of  the  timely  efforts  of  Dr.  Pallett's  group.  NIST  has  played  pivotal 
and  often  seminal  roles  in  the  development  and  distribution  of  several  key  corpora  to 
nurture  the  technology  effort.  To  wit,  here  are  some  key  ones:  TI-MIT  Acoustic  Phonetic 
Data  Base  (The  first,  true  speaker  independent  recognition  data  base  of  phonetically 
balanced  sentences),  Resource  Management  data  base  (task  specific  data  where  the 
constraints  of  the  task  can  be  taken  advantage  oO,  Air  Travel  Information  System  (ATIS) 
corpora  (the  first  spoken  language  technology  corpora  in  the  worid  that  has  given  birth  all 
state  of  the  art  speech  understanding  systems  including  those  at  CMU,  MIT,  BBN,  SRI 
and  NIST  itself),  and  SWITCHBOARD,  a  conversational  telephonic  corpora  created  by 
collaboration  between  NIST  and  TI  and  used  by  several  NSA  and  DARPA  contractors.  In 
fact,  SWITHCBOARD  has  been  valuable  for  developing  technologies  suitable  for 
telecommunication  applications  such  as  TI's  commercial  offerings  of  Voice  Dialing  at  Bell 
Atlantic,  and  Sprint." 
(Texas  Instruments) 

"The  Speech  recognition  research  program  at  IBM  has  benefited  significantly  from  the 
activities  of  NIST  in  the  area  of  standardized  test  procedures  and  coordination  of  Sf)eech 
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data  collection  and  dissemination.  IBM  has  had  a  long  history  of  research  in  speech 
recognition.  Our  research  program  in  the  early  '90s  was  focused  mainly  on  isolated  word 
dictation  systems  and  speaker  dependent  continuous  speech  recognition.  When  research 
focus  within  IBM  shifted  to  speaker  independent  large  vocabulary  continuous  speech 
recognition,  the  availability  of  reference  training  and  test  material  provided  by  NIST  was  an 
important  factor  in  helping  us  develop  technology  for  this  in  a  timely  fashion. 

By  providing  standardized  test  procedures  and  administering  system  benchmark  tests  on 
an  annual  basis,  NIST  has  contributed  significantly  to  the  development  of  speech 
recognition  technology  not  only  within  IBM,  but  also  in  the  continuous  speech  recognition 
community  at  large.  NIST's  efforts  in  this  area  have  helped  evaluate  the  relative  merits  of 
different  techniques  in  speech  recognition,  contributing  significantly  to  the  advances  in  the 
field.  Recently,  NIST  has  played  an  important  role  in  the  collection  and  dissemination  of 
data  from  broadcast  radio  sources.  IBM's  research  program  has  again  benefited  from  this 
data,  enabling  us  to  improve  the  performance  of  speech  recognition  systems  for 
transcribing  radio  broadcast  news  data." 
(IBM) 
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Mrs.  MORELLA.  Thank  you,  Mr.  Brownstein.  It's  an  honor  to  have 
you  among  my  constituency.  I  always  say  they're  so  intelhgent,  so 
well-informed.  They're  leaders  in  the  nation,  right,  Ms.  McCarthy? 

[Laughter.] 

Mrs.  MORELLA.  In  fact,  I  must  say,  the  three  of  you  have  very 
succinctly  presented  testimony.  I  think  we've  kind  of  set  a  record 
in  terms  of  time,  and  again,  I  repeat  to  all  of  you  that  the  totality 
of  your  statements,  because  I  notice  that  you  added  more  material, 
will  be  in  the  record. 

I  will  start  off  with  a  couple  of  questions,  turn  it  over  to  Ms. 
McCarthy,  and  then  get  back  to  you  again  with  some  more  ques- 
tions. 

NIST  is  merging  two  existing  laboratories  to  form,  as  you  have 
mentioned,  the  Information  Technology  Laboratory  to  achieve  a 
stronger  focus  on  the  information  technology  industry.  One  might 
wonder  how  NIST  can  impact  an  intensely  competitive  industry 
that  already  spends  about  $30  billion  a  year  in  research  and  devel- 
opment. 

So  I  just  wonder,  gentlemen,  how  do  you  select  research  areas  to 
assure  that  NIST  has  the  greatest  impact  on  the  information  tech- 
nology industry  and  the  greatest  social  return  for  the  Federal  in- 
vestment? It  is  a  tough  thing  to  do,  but  are  there  certain  criteria 
standards  that  are  used? 

Dr.  Wakid.  We  work  very  closely  with  industry  on  identifying  the 
priorities  of  their  needs  and  we  take  a  very  iinique  role  that  indus- 
try itself  does  not  provide,  that  is,  the  development  of  tests  to  es- 
tablish the  merit,  the  commercial  merit  of  research  areas.  Our  pri- 
orities are  truly  determined  by  our  stakeholders. 

Mrs.  MoRELLA.  By  the  stakeholders? 

Dr.  Wakid.  Our  industrial — meaning  industry. 

Mrs.  MORELLA.  Do  they  have  a  strong  role  to  play? 

Dr.  Wakid.  When  competing  industrial  researchers  are  trying  to 
evaluate  the  merits  of  their  work,  they  tell  us  to  develop  tests  that 
are  open  and  pre-competitive  that  they  could  use  to  cross-evaluate 
the  research.  So  they  use  us  as  a  neutral  technical  ground  to  estab- 
lish the  conmmercial  merit  of  their  work. 

Mrs.  MORELLA.  How  does  NIST  assure,  if  there  can  be  any  assur- 
ances, that  they  are  not  duplicating  or  preempting  activities  that 
would  otherwise  have  occurred  in  the  commercial  sector? 

Dr.  Wakid.  What  we  do  is  pre-commercial,  so  the  research  activi- 
ties that  we  broker  or  the  pre-standards  or  even  standardization 
activity  that  we  technically  broker  do  not  have  any  replicated  mar- 
ket existence. 

Mrs.  MoRELLA.  Would  either  of  the  other  two  panelists  like  to 
comment  on  that?  If  not,  I  will  go  into  another  question.  Do  you 
think  he  handled  it  adequately? 

There  was  discussion  about  the  importance  of  standards  in  the 
information  technology,  the  importance  to  industry,  the  Federal 
Government,  and  importance  also  to  the  consumer.  We  know  that. 
Yet,  Dr.  Wakid  indicated  that  the  new  Information  Technology 
Laboratory  will  move  from  the  development  of  standards  to  the  de- 
velopment of  conformance  testing  and  test  materials.  I  wondered  if 
you'd  like  to  comment  on  what  the  reason  is  for  that  shift  is  in  em- 
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phasis,  or  reasons.  There  is  probably  more  than  one  reason  for  that 
shift. 

Dr.  Brownstein.  I  cannot  comment  directly,  not  having  been 
part  of  the  planning  process. 

Mrs.  MORELLA.  Right. 

Dr.  Brownstein.  I  think  that  is  a  very  healthy  movement,  how- 
ever. The  marketplace  speaks  rapidly  these  days  about  what  it  will 
accept  as  a  standard  and  it  turns  out  that  there  are  any  number 
of  organizations  that  work  on  standards.  I  think  NIST,  by  having — 
or  the  labs,  by  having  a  reasonable  representation  and  continu- 
ously scanning  what  is  going  on  in  some  of  those  standards  proc- 
esses has  to  come  by  some  process  to  decide  where  it  is  going  to 
participate. 

Our  members  pointed  out  several  areas  which  are,  in  fact,  very 
forward-thinking  for  looking  at  areas  that  are  also  very  timely  and 
difficult  to  deal  with  where  they  have  fallen,  that  is,  NIST  has  fall- 
en into  a  happy  role  as  the  neutral  party  that  would  be  trusted  in 
providing  performance  evaluations  so  that  the  industry,  for  exam- 
ple, that  are  building  cable  modems,  and  make  the  collective  deci- 
sion about  where  they  want  to  go.  I  think  that  is  a  good  example 
of  NIST  moving  into  an  appropriate  role  in  a  contemporary  issue 
area. 

The  broader  issue  of  how  to  select  areas  in  which  to  work  and 
which  ones  to  stay  in,  I  think,  is  well  deserving  of  some  systematic 
planning  activity  on  the  part  of  the  labs  and  I  would  just  defer  to 
Dr.  Wakid  to  tell  you  about  the  processes  they  use  because  I  have 
not  been  involved  in  them. 

Mrs.  MORELLA.  And  in  so  doing,  maybe  Dr.  Roskies  would  like 
to  comment.  Is  the  decision  making  process  concerning  NIST's  par- 
ticipation in  the  voluntary  standards  process,  is  it  open  to  all 
stakeholders,  including  those  in  industry? 

Dr.  Roskies.  Would  you  repeat  that? 

Mrs.  Morella.  I  just  wondered  if  in  the  decision  making  process 
concerning  NIST  participation  in  the  voluntary  standards  situa- 
tion, is  it  open  to  all  stakeholders,  including  those  in  industry?  I 
think  Dr.  Wakid  would  probably  like  to  comment  affirmatively  on 
that,  would  you? 

Dr.  Wakid.  Mrs.  Morella,  we  do  not  develop  standards.  Industry 
develops  standards. 

Mrs.  Morella.  Right. 

Dr.  Wakid.  We  simply  participate  and  help  out.  But  our  key  role 
is  the  development  of  conformance  tests  for  standards  because  the 
conformance  tests  capture  the  technical  underpinning  of  these 
standards.  As  soon  as  a  standard  starts  churning  in  the  market- 
place, we  just  simply  use  this  to  say,  let's  go  on  to  the  next  priority, 
homework  on  our  list. 

So  when  we  participate  in  standards  committees,  they're  always 
open.  These  are  voluntary  standards  committees.  They're  open.  So 
the  openness  of  the  process  is  provided  inherently  by  the  standard 
development  process  and  our  participation  is  simply  one  of  getting 
engaged  in  an  open  forum. 

Dr.  Roskies.  If  I  could  make  a  comment  on  your  previous  ques- 
tion, I  just  wanted  to  say  that  the  formal  standards  process  seems 
to  move  too  slowly  in  the  information  technology  arena  to  be  useful. 
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So  moving  from  developing  standards  to  conformance  testing  is  a 
recognition  of  where  NIST  can  play  a  useful  role.  Otherwise,  you 
were  in  the  situation  where  the  formal  standards  bodies  were  sim- 
ply catching  up  to  the  actual — what  the  world  was  doing  and  there 
was  not  much  point  to  it. 

Mrs.  MORELLA.  So  it  expedites  the  process,  helps  to  accelerate  it, 
sort  of.  Will  NIST  reduce  its  involvement  in  helping  to  set  vol- 
untary standards  in  the  information  technology  industry,  or  is  Dr. 
Roskies'  answer  in  part  indicating  that  it  will  be  moving  ahead 
faster  by  that  kind  of  coordination? 

Dr.  Wakid.  We  have  no  intention  of  reducing  our  involvement, 
but  we  do  always  select  advanced  areas  to  be  engaged  in.  Again, 
since  information  technology  is  a  very  fast-moving  technology,  we 
have  to  keep  up,  helping  with  pushing  the  envelope  so  that  our  in- 
dustry is  competitive.  To  do  so,  we  have  to  focus  on  the  advanced 
area  and  not  on  the  areas  that  are  close  to  the  marketplace. 

Mrs.  MORELLA.  Ms.  McCarthy,  we  will  give  you  a  chance  to  ques- 
tion as  soon  as  you  get  back. 

Mr.  Tanner,  we  recognize  you  for  any  questioning  of  this  first 
panel. 

Mr.  Tanner.  Thank  you.  I  am  sorry  I  had  to  step  out.  I  apolo- 
gize. I  have  read  over  your  testimony,  though. 

As  you  maybe  know,  earlier  this  year,  the  subcommittee  held  a 
hearing  on  the  year  2000  problem.  I'd  be  interested  in  your  assess- 
ment as  to  the  scope  of  the  problem,  what  you  think  the  private 
sector  is  doing  to  address  the  issue,  what  is  NIST  doing  in  this 
area,  and  what,  if  any,  recommendation  would  you  have  for  this 
Committee  or  Congress  in  this  matter? 

Dr.  Wakid.  We  have  published  a  Federal  standard  quite  some 
time  ago,  a  few  years  ago,  to  explain  the  problem  and  to  explain 
how  technically  to  solve  the  problem.  It  has  been  a  very  old  Federal 
standard.  Very  recently,  we  put  on  the  Web  a  description  of  how, 
again  technically,  this  problem  can  be  solved,  and  everybody  has 
access  to  our  Web  to  get  guidelines. 

The  actual  process  of 

Mr.  Tanner.  May  I  interrupt?  How  many  hits  have  you  had  on 
that  one,  do  you  know? 

[The  following  response  was  received  for  the  record:] 

The  Year  2000  home  page  on  the  Internet  was  put  on-line  on  May  2,  1996.  As 
of  June  26,  1996,  the  home  page  has  been  accessed  290  times. 

Dr.  Wakid.  My  apology.  I  do  not  have  this  number  on  me.  I  do 
want  to  explain  that  industry  has  the  same  problem,  just  as  gov- 
ernment agencies.  The  problem  is  generic,  all  across.  The  actual 
process  of  going  through  legacy  code  and  making  changes  to  honor 
the  century  field  is  an  extremely  tedious,  expensive  problem,  again, 
for  industry  and  for  the  government.  So  the  actual — again,  the  ac- 
tual process  of  maintaining  legacy  code  that's  not  documented  that 
the  programmers  that  wrote  the  code  have  left  companies,  what 
have  you,  is  a  very  serious,  a  very  significant,  very  tedious  tech- 
nically a  lightweight  problem,  if  you  will. 

I  truly  do  not  know  specifically  what  to  recommend  to  this  com- 
mittee other  than  the  fact  that  we  are  sponsoring  a  big  inter- 
national workshop  to  educate  users  on  the  problem  and  to  allow 
people  who  have  confronted  this  problem  to  share  with  others  case 
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studies  of  how  you  go  about  addressing  this  problem.  This  is  our 
next  step.  We  are  running  an  international  workshop,  again,  for 
promotion  of  solutions. 

Mr.  Tanner.  I  appreciate  that.  I  am  afraid  that  only  the  people 
who  are  aware  and  who  are  working  on  it  and  are  going  to  do 
something  about  it  will  show  up  at  the  workshop,  but  there's  noth- 
ing you  can  do  about  that. 

What  would  be  the  wisdom  of  NIST  developing  some  sort  of  per- 
formance test  or  some  sort  of  drill  to  test  some  of  the  theories  that 
are  floating?  Would  that  be  of  any  use? 

Dr.  Wakid.  I  think  so.  I  think  we're  very  willing  to  establish 
some  kind  of  test  for  the  integrity  of  a  code  that's  been  modified, 
if  you  will.  I  think  that's  fine,  yes,  sir. 

Mr.  Tanner.  Do  either  of  you  two  gentlemen  have  anything  to 
add? 

Dr.  ROSKIES.  No.  I  think  it  is  a  good  idea  to  look  into  this. 

Mr.  Tanner.  That  is  all  I  have.  Madam  Chairwoman.  Thank  you. 

Mrs.  MORELLA.  I  don't  know  whether  Mr.  McHale  has  had  a 
chance  to  catch  his  breath.  This  is  the  second  panel  on  information 
technology,  if  you  would  like  to  ask  them  any  questions. 

Mr.  McHale.  Madam  Chairwoman,  I  would  be  disingenuous,  not 
that  that's  ever  stopped  me  before,  were  I  to  thank  the  witnesses 
for  the  testimony  that  I  did  not  hear.  On  the  assumption  that  it 
was  wonderfully  informative  and  eloquent,  I  thank  you.  In  light  of 
the  fact  that  I  arrived  in  the  room  ten  seconds  ago,  I  do  not  have 
any  questions.  Thank  you. 

Mrs.  Morella.  I  guess  I  will  pick  up  on  a  few  questions,  too.  The 
effect  of  the  passage  of  the  Telecommunications  Act  of  1996  is  one 
I'm  curious  about.  It's  going  to  bring  about  substantial  change  in 
our  U.S.  telecommunications  industry.  For  example,  the  Act's  pro- 
visions will  remove  previous  regulatory  restrictions  delineating  the 
type  of  research  that  some  industry  laboratories,  such  as  Bellcore 
and  Bell  Labs,  can  undertake,  and  by  some  accounts,  these  two  lab- 
oratories have  performed  80  percent  of  the  R&D  in  the  United 
States  telecommunications  industry. 

So  I  wonder,  how  will  the  restructuring  of  the  telecommuni- 
cations industry  affect  the  work  that  NIST  does?  Have  you  had  a 
chance  to  think  about  that?  Then  I  wonder  if  NIST  should  then  ask 
for  a  direct  industry  support  for  its  basic  and  applied  research  pro- 
grams in  information  technology  as  the  telecommunications  indus- 
try moves  closer  to  the  commercial  activities.  Dr.  Brownstein? 

Dr.  Brownstein.  The  effect  that  I've  seen  and  the  effect  that  our 
members  have  talked  about  is  that  industry  laboratories  are  deal- 
ing more  with  development  and  less  with  research,  more  with — 
have  a  greater  concern  for  products  in  the  marketplace,  getting 
those  products  into  the  marketplace,  and  far  less  of  a  concern  with 
out-year  intellectual — development  of  intellectual  resources  for  fu- 
ture products. 

That  makes  them  more  interested  in  seeing  organizations  like 
NIST  and  other  parts  of  the  Federal  Government  supporting  basic 
research  and  plajdng  the  third  party  sort  of  neutral  host,  convener, 
£ind  assister  role  that  NIST  has  very  well  carved  out,  this  labora- 
tory has  very  well  carved  out.  I  would  expect  that  you  will  see 
strong  interest  on  the  part  of  industry  in  supporting  those  parts  of 
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NIST  activities  that  meet  its  needs.  That  will  be  up  to  sort  of  cre- 
ative management  in  NIST  and  creative  management  in  the  gov- 
ernment to  find  ways  to  work  with  industry. 

The  Cross-Industry  Working  Team  was  created  specifically  to 
allow  industry  to  get  together  and  talk  about  issues  that  were  be- 
yond immediate  product  cycles,  something  that  it  had  never  really 
done  before.  We  have  all  competitors  engaging  in  kind  of  a  strate- 
gic assessment  of  why  things  work  £ind  how  some  integrity  could 
be  maintained  in  the  future. 

There  is  one  non-private  sector  organization  in  the  Cross-Indus- 
try Working  Team,  and  that  is  NIST.  When  NIST  applied,  there 
were  major  debates  about  whether  we  should  allow  a  government 
agency  to  participate.  In  the  end,  it  came  down  to  the  same  reason 
that  we  allow  any  participant  to  join — we  invite  people  to  join;  we 
are  not  open  to  just  anyone  coming  in — and  that  was  because  we 
thought  they  had  a  real  contribution  to  make  and  very  much  want- 
ed to  find  a  creative  way  to  be  able  to  work  with  the  leadership 
at  NIST  in  understanding  what  industry  concerns  would  be  several 
years  out. 

So  I  think  that's  a  very  healthy  sign,  that  NIST  understands  this 
new  environment,  and  I  can  tell  you  that  industry  understands  this 
new  environment  very  well.  They've  been  planning  for  it  for  some 
time  and  I  think  you're  going  to  see  a  large  amount  of  industry 
support  for  a  continued  role  of  government  in  a  variety  of  informa- 
tion technology  areas  that  stay  out  of  the  immediate  marketplace 
competition. 

Mrs.  MORELLA.  That  is  an  interesting  response. 

Dr.  Roskies? 

Dr.  Roskies.  Yes,  I  would  like  to  add,  the  panel  which  I  chair 
conducts  an  annual  assessment  of  what's  going  on  at  NIST  in  the 
two  laboratories  and  one  of  the  questions  that  we  ask  of  NIST 
management  every  time  is  what  is  the  value  added  of  NIST.  I  don't 
think  it's  easy  to  give  a  set  of  simple  rules  that  will  allow  NIST 
to  pick  projects  that  don't  substantially  overlap  activities  that  go  on 
in  other  places,  but  we  have  on  a  case-by-case  basis  an  analysis 
every  year.  The  management  is  doing  analysis  even  more  than 
that,  but  we  get  a  chance  every  year  to  review  this  and  one  of  the 
questions  we  ask  is  whether  there  is  sufficient  value  added  on 
NIST's  part. 

So  they  have  to  be  aware  of  this  problem.  It's  easy  to  ask  the 
question.  It's  not  so  easy  to  give  guidance  on  how  to  choose  those 
projects  so  that  there  isn't  substantial  overlap. 

Mrs.  MoRELLA.  So  do  they  respond  with  what  they  consider  to 
be  the  value  that  was  added? 

Dr.  Roskies.  Yes. 

Mrs.  Morella.  They  explain  that  and  what  they  will  do  further 
to  enhance  it? 

Dr.  Roskies.  Yes. 

Mrs.  Morella.  Do  you  want  a  shot  at  that,  Dr.  Wakid? 

Dr.  Wakid.  I'd  like  really  to  answer  a  different  spin  to  your  ques- 
tion about  telecommunications. 

Mrs.  Morella.  Sure. 

Dr.  Wakid.  As  you  know,  with  the  digital  technology,  there  is  a 
convergence  between  telecommunications  and  computing  and  even 
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display.  These  technologies  are  convergent.  And  so  in  the  future,  as 
they  say,  a  bit  is  a  bit.  If  you're  using  it  for  voice  or  for  a  computer 
transaction  or  for  a  video,  it's  just  a  bit.  The  bit  streams  are  all 
the  same.  So  technology  is  converging. 

The  fact  that  very  big  think  tanks,  national  think  tanks  who  are 
very  impressive,  such  as  Bellcore,  are  up  for  sale,  for  example,  does 
really  mean  there  is  going  to  be  more  and  more  national  need  for 
a  lab  such  as  NIST  to  do  more  work.  To  me,  this  is  very  obvious 
because  the  owners  of  Bellcore  have  been  seven  regional  telephone 
companies  that  don't  have  a  central  think  tank  anymore  and  I 
think  the  nation  does  need  one. 

Mrs.  MORELLA.  Thank  you. 

Before  I  ask  any  other  questions,  I  want  to  turn  it  over  to  Ms. 
McCarthy  for  any  questioning  she  may  have  of  this  panel. 

Ms.  McCarthy.  Thank  you,  Madam  Chairwoman,  and  with  the 
indulgence  of  Mr.  Tanner,  I  will  proceed,  because  I'd  like  to  visit 
with  you  just  a  little  bit  about  the  development  of  the  encryption 
standards.  I  know  we're  in  a  debate  about  the  national  interests 
and  a  need  to  balance  that  with  law  enforcement  interests  and 
other  aspects  of  the  public  interest,  but  this  is  going  to  be  pretty 
critical  in  the  future  for  U.S.  companies  and  certainly  for  all  civil 
liberties  and  open  government. 

I  thought  if  you'd  just  reflect  on  the  opportunities  that  are  being 
provided  for  our  private  sector  in  the  process  through  which  these 
encryption  standards  are  being  developed,  that  would  be  of  interest 
to  me,  I  know,  and  I  think  to  other  Members  on  the  Subcommittee. 

Dr.  Wakid.  You  are  asking  me  the  question? 

Ms.  McCarthy.  Whichever,  all  of  you  or  any  one  of  you. 

Dr.  Wakid.  Our  role  is  to  help  government  agencies  with  the 
technical  protection  of  unclassified  but  sensitive  information.  The 
question  you  really  asked  is  way  above  my  civil  service  grade  and 
I  would  refer  you  to  the  Interagency  Working  Committee  of  the  Na- 
tional Security  Council  that  really  can  answer  your  policy  question. 
We  do  not  deal  with  policy  questions  of  the  nature  that  you  asked. 

Ms.  McCarthy.  So  they  are  the  ones  who  decide  about  private 
sector  involvement  in  the  development  of  these? 

Dr.  Wakid.  Yes. 

Ms.  McCarthy.  You  are  just  one  facet  of  it  and  have  no  way  in 
that,  nor  would  you  listen  to  private  sector  and  their  concerns. 

Dr.  Wakid.  We  are  a  technical  broker,  if  you  will,  but  purely 
technical,  non-policy.  We  are  a  technical  broker  between  the  pri- 
vate sector  and  Federal  needs,  but  we're  not  a  policy  broker  at  all. 

Ms.  McCarthy.  I'm  reminded  that  under  the  Computer  Security 
Act,  NIST  is  charged  with  broad  responsibilities  for  the  protection 
of  sensitive  but  non-classified  information  on  Federal  computer  sys- 
tems. In  discharging  those  responsibilities,  NIST  has  been  criti- 
cized for  focusing  on  picking  technologies  for  standards  such  as  the 
product-oriented  escrowed  encryption  standard.  Why  is  NIST  not 
being  more  policy  neutral  in  that,  given  what  you  have  just  ex- 
plained to  the  subcommittee? 

Dr.  Wakid.  What  the  Security  Act  gives  us  is  a  mandate  to  pub- 
lish Federal  standards  for  Federal  agencies  to  protect  their  unclas- 
sified but  sensitive  information,  so  this  is  a  purely  Federal  Act  for 
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government  agencies  who  are  our  stakeholders  and  we're  perform- 
ing this  technical  mission. 

But  the  policy  question  you  asked,  I  thought,  has  more  to  do  with 
the  balance  between  industry  export  needs,  and  law  enforcement 
needs  and  national  security  needs.  The  balance  of  these  needs  and 
the  policy-related  issues  of  these  needs  are  way  above  my  civil 
service  grade  is  honestly  what  I'm  telling  you. 

Ms.  McCarthy.  So  NIST  will  have  no  say  in  those  decisions? 

Dr.  Wakid.  That  is  correct. 

Ms.  McCarthy.  What  is  the  role,  then,  of  NIST  in  coordinating 
Federal  responses  to  things  like  unauthorized  access  to  Federal 
computer  systems? 

Dr.  Wakid.  We  do  help  a  lot  with  this.  Again,  we  provide  tech- 
nical protection  and  technical  consulting  for  building  fire  walls.  We 
are  really  very  active  in  this  area  of  protecting  government  infor- 
mation. This  is  a  technical  role  that  we  have  been  very  glad  to  help 
out  on. 

Ms.  McCarthy.  And  in  developing  that  role,  what  outreach  did 
you  do  in  order  to  make  sure  you  were  choosing  the  right  re- 
sponses? 

Dr.  Wakid.  We  tried  to  understand  how  the  hackers  intrude  into 
various,  be  it  private  or  government  networks,  and  from  the  tech- 
nical know-how,  we  tried  to  build  protection  mechanisms.  So  we 
very  strongly  interact  with  the  private  sector  in  terms  of  learning 
what  the  facts  are  so  we  can  protect  our  agencies  better. 

Ms.  McCarthy.  Those  Federal  responses  that  you've  just  de- 
scribed development  of,  are  those  the  kinds  of  models  that  the  pri- 
vate sector  might  want  to  also  emulate? 

Dr.  Wakid.  Yes,  that's  a  very  good  question.  Yes,  we  like  to  do 
things  that  are  of  common  interest  to  both,  again,  technical  com- 
mon interest  to  both,  both  meaning  the  public  and  private  sector, 
yes. 

Ms.  McCarthy.  And  in  so  doing,  do  you  then  listen  to  the  pri- 
vate sector  in  the  development  of  those? 

Dr.  Wakid.  Yes,  we  do,  but  the  issues  related  to  standards  for 
balance  of  needs,  as  you  originally  asked,  again,  we're  not  really 
the  decision  makers  there. 

Ms.  McCarthy.  So  at  some  level,  it's  okay  to  get  that  private 
sector  input,  but  not  at  others? 

Dr.  Wakid.  Yes. 

Ms.  McCarthy.  And  who  decides  where  that  line  is? 

Dr.  Wakid.  The  Interagency  Working  Team.  This  is,  again,  a 
very  high-level  committee  that's  part  of  the  National  Security 
Council.  It's  a  very  senior  committee  and  the  question  you  asked 
is  truly  above  NIST,  way  above  NIST. 

Ms.  McCarthy.  I  appreciate  the  role  that  NIST  is  playing  and 
I  feel  very  strongly  that  this  is  a  subject  of  major  interest  through- 
out our  nation  and  certainly  the  world  and  something  that  we  need 
to  focus  on  and  we  need  to  be  sure  that  as  we  develop  those  stand- 
ards that  we  do  so  with  input  from  all  affected  as  best  we  can. 

Dr.  Wakid.  Thank  you. 

Ms.  McCarthy.  Madam  Chair,  it  looks  like  Dr.  Brownstein 
wanted  to  comment. 
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Dr.  Brownstein.  This  could  easily  be  the  subject  of  several  more 
hearings.  The  Cross-Industry  Working  Team  was  asked  last  year 
by  the  Interagency  Committee  of  the  government  to  take  a  look  at 
the  process  by  which  information  technology  security  policy  is  set, 
not  the  specific  policies  but  the  process,  to  evaluate  its  adequacy 
and  to  make  recommendations.  We  did  that  in  a  report,  which  has 
been  made  public,  signed  by  45  companies.  It  was  accepted  by  the 
committee  that  asked  for  it,  which  has  taken  many  of  its  rec- 
ommendations and  folded  them  into  a  set  of  recommendations  and 
is  proceeding  on  at  the  levels  above  Dr.  Wakid's  pay  grade  to  try 
to  change  some  of  the  way  the  policy  is  set. 

But  it  is  perceived  as  an  issue,  not  just  on  the  part  of  govern- 
ment, because  so  many  agencies  have  an  interest  in  this,  but  a 
problem  for  industry,  because,  in  fact,  there  is  not  a  single  industry 
here  that  speaks  with  a  single  voice  on  the  matter.  It's  a  complex 
matter  and  I'd  invite  you  to  take  a  look  at  the  report. 

Ms.  McCarthy.  I  would  welcome  an  opportunity  to  review  the 
document. 

Dr.  Brownstein.  I'll  leave  you  a  copy. 

Ms.  McCarthy.  Madam  Chairwoman,  thank  you  for  your  indul- 
gence on  this.  I  know  that  Representative  Laughlin  and  others  on 
your  subcommittee  have  interest  in  this  issue,  so  perhaps  there 
could  be  a  future  hearing  to  review  some  of  these  questions  that 
are  being  posed  in  our  districts. 

Mrs.  MORELLA.  And  I  know  it's  the  topic  of  legislation  that  is  be- 
fore another  committee.  The  encrjrption  concept  is  critically  impor- 
tant, so  we'll  check  on  that  jurisdiction. 

I  just  want  to  finally  ask  you  about  international  standards.  Are 
you  satisfied  with  the  present  process  for  international  standard 
setting  and  the  NIST  role  to  assure  that  our  U.S.  trade  interests 
are  properly  addressed? 

Dr.  Wakid.  This  is  very  much  the  core  of  what  we  do.  In  fact, 
it's  a  very  key  selection  criteria  for  the  programs  or  the  standards 
we  decide  to  take  on.  If  such  standards  have  a  potential  payoff  on 
large  markets,  and  especially  international  trade,  that  will  rank 
them  very  high  on  our  selection  criteria.  If  we  want  to  take  on  a 
technology  that  has  an  international  trade  impact  then  for  apples 
and  apples,  that  technology  will  move  up  on  our  priority  list.  The 
answer  is  a  big  yes. 

Mrs.  MORELLA.  Since  our  other  two  panelists  have  probably  had 
some  experience  in  that  area,  would  they  like  to  comment?  Do  you 
feel  that  this  answer  is  adequate?  Should  there  be  some  changes? 

Dr.  Brownstein.  I  would  like  to — I  don't  know  how  adequate  or 
inadequate  it  is,  frankly,  but  I'd  like  to  second  what  I  just  heard. 
The  U.S.  is  perhaps  the  only  country  that  does  not  have  a  strategic 
single  representative  that  goes  on  and  works  in  its  own  interest  in 
international  standards.  We  have  separate  sectors,  public  sector, 
private  sector.  We  heard  from  our  members  that  they  like  the  role 
that  NIST  plays,  but  I  think  that's — and  frankly,  having  been  in- 
volved in  some  of  those  activities  over  time,  I  can  assure  you  we're 
better  off  with  technical  people  at  NIST  pla3dng  those  roles  than 
we  are  with  diplomatic  people  attempting  to  play  them.  I  would 
just  think  that  that's  an  area  of  strong  interest  in  industry  and 
worthy  of  further  consideration. 
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Mrs.  MORELLA.  Dr.  Roskies  agrees. 

Dr.  Roskies.  Right. 

Mrs.  MORELLA.  I'm  going  to  turn  the  chairwomanship  into  the 
chairmanship  with  Mr.  Gutknecht  as  I  go  over  to  the  Senate  to  in- 
troduce a  nominee  for  our  Federal  Court  system.  Mr.  McHale  has 
not  had  a  chance  to  ask  any  questions.  I  thank  you. 

Mr.  Gutknecht.  (Presiding)  Thank  you. 

Mr.  McHale? 

Mr.  McHale.  Mr.  Chairman,  thank  you. 

Gentlemen,  I'm  not  sure  if  any  of  you  are  familiar  with  the  rela- 
tionship that  NIST  has  had  in  the  past  with  AT&T,  and  so  perhaps 
my  question  is  going  a  little  bit  far  afield,  but  AT&T,  or  what  had 
been  AT&T,  now  broken  down  into  independent  elements,  had  a 
strong  historic  presence  in  my  district.  Lucent,  one  of  the  subordi- 
nate elements  of  what  had  been  AT&T,  has  a  very  large  facility  in 
my  district. 

Can  you  give  me  just  a  brief  sketch  of  NIST's  past  relationship 
with  AT&T,  and  in  light  of  the  breakup  of  AT&T,  do  you  anticipate 
any  change  in  that  relationship  with  regard  to  what  had  been  one 
company  now  broken  down  into  component  parts? 

Dr.  Wakid.  Would  you  like  me  to  answer  this? 

Mr.  McHale.  Please. 

Dr.  Wakid.  We've  had  a  very  old,  ongoing  relationship  with 
AT&T.  We  formed  a  consortium  through  a  NIST  CRADA— you 
have  heard  of  CRADAs — where  AT&T  was  one  of  the  very  key 
players,  as  well  as  some  RBOCs,  as  well  as  some  computer  compa- 
nies and  users.  This  U.S.  consortium,  that  was  sponsored  by  NIST, 
developed  tests  for  ISDN,  Integrated  Services  Digital  Network, 
under  our  leadership. 

Mr.  McHale.  I'm  sorry,  would  you  say  that  again? 

Dr.  Wakid.  They  developed  tests,  tests  mean  conformance 
tests 

Mr.  McHale.  Yes. 

Dr.  Wakid.  (continuing)  — that  we  took  as  a  U.S.  contribution  to 
the  international  standards  arena  and  they  turned  out  to  be  inter- 
national standards  with  our  very  strong  involvement  in  this.  That 
gave  AT&T  and  our  companies  a  big  edge  in  the  international  mar- 
ket because  international  buyers  are  using  international  standards 
that  conform  to  this  consortium  output. 

We  have  been  very  involved  with  AT&T,  just  like  with  other 
companies,  on  the  developing  of  reliable  speech  recognition  prod- 
ucts using  the  tests  that  I  was  talking  about,  and  as  a  matter  of 
fact,  AT&T  has  used  those  tests  to  develop  a  speech  recognition 
system  that  they  deployed  in  their  network  to  automate  operator 
services.  The  claim  is  made  by  AT&T  that  they  save  $300  million 
a  year  because  when  you  talk  to  an  operator  now,  you  are  talking 
to  a  computer  in  AT&T.  The  NIST  tests  were  very  key  in  making 
the  recognition  systems  more  reliable. 

We  have  other  numerous  examples  with  Bell  Labs  and  their  re- 
searchers, and  I  do  not  believe  that  the  breakup  of  AT&T  is  going 
to  be  an  impediment  to  our  ongoing  relationship. 

Mr.  McHale.  When  I  asked  my  question,  I  didn't  realize  that  I 
was  opening  the  door  to  such  a  success  story  central  to  the  mission 
historically  of  NIST,  and  obviously  providing  several  examples  of 
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where  that  kind  of  cooperation  in  the  private  sector  has  produced 
some  very  positive  commercial  results. 

Have  you,  since  the  breakup  of  AT&T,  has  NIST  proactively 
taken  any  steps  to  guarantee  an  ongoing  positive  relationship  with 
the  component  parts  of  what  had  been  AT&T,  and  specifically 
Lucent?  I  have  a  strong  interest  with  regard  to  that  company.  If 
not,  let  me  encourage  you  to  do  so. 

Dr.  Wakid.  Yes,  I  would — I  am  an  ex-employee  of  Bell  Labs,  so 
I  think  I  know  the  ropes  inside  AT&T. 

Mr.  McHale.  I  should  also  point  out.  Bell  also  has  a  strong  pres- 
ence in  my  district.  I'm  evenhandedly  parochial. 

[Laughter.] 

Dr.  Wakid.  No  problem  at  all. 

Mr.  McHale.  Thank  you,  Mr.  Chairman. 

Mr.  GUTKNECHT.  Thank  you. 

I  apologize  for  being  late.  I  don't  have  any  questions,  and  I  would 
like  to  thank  the  panel  for  joining  us. 

Does  anyone  else  have  any  questions?  Mr.  Tanner? 

Mr.  Tanner.  No  questions. 

Mr.  GUTKNECHT.  I'd  like  to  thank  the  panel  and  we'll  call  up  the 
panel  who  was  listed  first  who  we'll  hear  from  second.  Thank  you. 

Listed  first  is  Dr.  Robert  Hebner.  Welcome  back  to  the  Sub- 
committee. We  will  allow  you  to  speak  first. 

STATEMENTS  OF  DR.  ROBERT  E.  HEBNER,  ACTING  DEPUTY  DI- 
RECTOR, NATIONAL  INSTITUTE  OF  STANDARDS  AND  TECH- 
NOLOGY, GAITHERSBURG,  MARYLAND;  DR.  V.  THOMAS 
RHYNE,  CHAIR,  NATIONAL  RESEARCH  COUNCIL  BOARD  ON 
ASSESSMENT  OF  NIST  PROGRAMS,  PANEL  FOR  ELEC- 
TRONICS AND  ELECTRICAL  ENGINEERING,  WASHINGTON, 
D.C.;  AND  DR.  JAMES  A.  GLAZE,  VICE  PRESIDENT,  TECH- 
NOLOGY, SEMICONDUCTOR  INDUSTRY  ASSOCIATION,  SAN 
JOSE,  CALIFORNIA 

Mr.  Hebner.  Thank  you,  Mr.  Chairman,  and  Members  of  the 
Subcommittee.  I  appreciate  the  opportunity  to  discuss  the  work  of 
the  Electronics  and  Electrical  Engineering  Laboratory  of  the  Na- 
tional Institute  of  Standards  and  Technology. 

In  my  brief  opening  statement,  I  would  like  to  highlight  five  at- 
tributes of  this  laboratory.  First  of  all,  the  work  of  the  laboratory 
is,  and  must  be,  laboratory-based.  The  laboratory  is  global  in  both 
its  activities  and  its  outlook.  The  work  of  the  laboratory  is  highly 
leveraged.  The  work  of  the  laboratory  is  unique,  and  the  laboratory 
has  a  very  strong  industry  focus. 

First,  let  me  give  you  an  example  of  what  I  mean  by  the  work 
is,  and  must  be,  laboratory-based.  Several  years  ago,  the  Nobel 
Prize  in  physics  was  given  to  Brian  Josephson  of  England,  and  sub- 
sequently, a  second  Nobel  Prize  was  given  to  Von  Klitzing  in  Ger- 
many, both  for  study  of  the  behavior  of  electrons  as  you  go  near 
absolute  zero,  how  they  interact  with  each  other  and  with  outside 
forces.  The  staff  at  NIST  took  those  Nobel  Prizing  winning  discov- 
eries, took  them  into  our  laboratory,  hardened  them,  understood 
them,  worked  with  them.  Today,  they  provide  the  cornerstones  for 
electrical  measurements  in  the  United  States  and  around  the 
world. 


71 

The  work  is  global.  Any  commerce  that  takes  place  requires  two 
key  elements.  One  is  a  monetary  system  and  the  second  is  some 
way  of  measuring  either  quantity  or  performance.  In  the  area  of 
electronics,  the  measurement  of  performance  is  a  continuing  and 
growing  challenge  and  that  measurement  has  to  be  available  for 
U.S.  companies  to  manufacture  products  here,  to  sell  them  here, 
and  to  sell  them  abroad. 

The  work  is  highly  leveraged,  and  I  would  like  to  underscore  this 
with  an  example  from  an  industry  which  is  undergoing  significant 
change  right  now,  the  electric  utility  industry.  In  this  country 
today,  about  $200  billion  worth  of  electricity  is  sold  each  year.  To 
bill  that  amount  of  money,  the  meter  manufacturers,  the  electric 
utilities,  the  States  perform  about  eight  million  calibrations  of  watt 
hour  meters.  NIST  provides  the  input  to  that  system  to  assure  that 
it  remains  accurate  by  providing  fewer  than  100  calibrations.  We 
do  fewer  than  100  calibrations.  We  support  more  than  eight  million 
industry  calibrations,  which  supports  the  equitable  billing  of  more 
than  $200  billion  worth  of  electricity  in  the  U.S. 

The  work  that  we  do  is  unique.  We  are  recognized  as  being  the 
nation's  measurement  laboratory.  We  are  an  independent  third 
party.  We  are  unbiased.  We're  trusted.  A  good  example  of  that 
comes  from  the  semiconductor  industry.  When  they  had  difficulty 
measuring  the  quality  of  the  silicon  wafers  that  were  being  bought 
several  years  ago,  they  came  to  us  and  asked  us  to  help  them  solve 
that  problem.  We  worked  with  the  industry  to  develop  standard 
measurement  techniques  and  to  look  at  how  the  measurements 
were  being  made.  They  wrote  their  industry  standards  based  on 
some  of  the  measurement  technology  we  had  developed. 

In  addition,  we  provided  standard  reference  materials,  which  we 
sold  to  anyone  who  wanted  to  buy  them  so  they  could  verify  that 
their  measurements  remain  consistent.  We  have  in  our  files  a  let- 
ter from  the  vice  president  of  a  small  company  which  thanked  us 
for  providing  these  because  he  claimed  they  had  $50  million  worth 
of  product  they  would  not  have  been  able  to  sell  had  they  not  had 
our  standard  reference  material  that  they  could  use  to  show  the 
major  suppliers  that  they  actually  could  do  the  measurements  that 
they  claimed  that  they  could  make. 

Finally,  we  have  an  industry  focus.  The  work  is  planned  in  co- 
ordination with  industry.  We  work  very  closely  with  industry  in 
carrying  out  the  program  and,  as  you  will  hear,  I  am  sure,  from 
the  others  on  the  panel,  the  work  is  assessed  by  industry  and  we 
get  strong  and  direct  feedback  as  to  how  well  we  are  doing. 

I  welcome  your  questions. 

[The  prepared  statement  of  Mr.  Hebner  follows:] 
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Madam  Chairwoman  and  Members  of  the  Subcommittee,  I  appreciate  the  opportunity  to  discuss 
the  work  of  the  Electronics  and  Electrical  Engineering  Laboratory  of  the  National  Institute  of 
Standards  and  Technology.  The  Electronics  and  Electrical  Engineering  Laboratory  (EEEL), 
through  its  technical  laboratory  research  programs,  supports  the  U.S.  electronics  industry,  its 
suppliers  and  its  customers,  including  the  Federal  govemment,  by  providing  the  measurement 
technology  that  US  industry  needs  to  improve  its  competitive  position  and  the  govemment 
needs  to  improve  efficiency  in  technical  operations. 

This  measurement  technology  is  required  for  industrial  research  and  development,  for 
improvements  in  the  design  and  manufacturing  of  quality  products,  for  proof  of  performance,  for 
mari^etplace  transactions  including  the  entry  of  U.S.  products  into  intemational  martlets,  and  for 
product  application.  The  measurement  technologies  resulting  from  this  program  support  the 
fundamental  electronic  technologies  of  semiconductors,  magnetics,  and  superconductors  that 
underlie  all  modem  electronic  systems,  electronic  components  technologies  for  information  and 
communication  aplications,  including  fiber  optics,  optoelectronics,  microwaves,  and  video; 
electrical  power  systems;  the  advanced  manufacturing  of  electronic  products;  and  electronic 
instrumentation. 

EEEL  provides  the  national  reference  standards  that  are  the  fundamental  basis  for  all  electrical 
measurements  in  the  U.S.  and  are  required  to  assure  the  accuracy  of  industry's  measurements. 
To  provide  consistency  and  accuracy,  all  measurements  must  ultimately  be  referred  back  to 
national  reference  standards.   For  example,  to  measure  silicon  resistivity,  a  semiconductor 
manufacturer  needs  calibrated  instruments  and  standardized  procedures  to  be  assured  of 
accurate  results.   In  our  efficient  industnal  system,  the  semiconductor  manufacturer  relies  on 
equipment  manufacturers  and  private  sector  calibration  laboratories  to  provide  calibrated 
instruments    To  certify  the  accuracy  of  the  instruments  being  manufactured,  the  instrument 
manufacturer  must  have  available  appropriate  standards  calibrated  in  terms  of  national  electrical 
standards.  The  instrument  manufacturer  either  maintains  these  and  has  them  compared 
regulariy  with  NIST  standards,  or  uses  a  private  sector  calibration  laboratory  which  provides  the 
reference  to  the  national  standards  maintained  by  NIST.    This  process  is  called  traceability. 

EEEL  is  the  linchpin  of  this  process  for  electrical  measurements.   It  maintains  all  the  national 
reference  standards  for  electricity  for  the  U.S.  and  provides  appropriate  calibration  services  and 
calibrated  artifacts  to  private  industry  and  other  govemment  agencies.   Realizing  national 
reference  standards  in  terms  of  their  internationally  agreed-upon  definitions  is  technically  difficult 
and  requires  a  dedicated  laboratory  capability.  NIST  has  that  legally-mandated  responsibility  for 
the  United  States. 

The  NIST  Laboratories  must  do  more  than  simply  maintain  standards  if  the  US  is  to  have  the 
necessary  measurement  systems.  The  second  (and  larger)  part  of  the  problem  is  the  necessity 
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of  having  available  suitable,  appropriate,  and  meaningful  measurement  methods  that  permit 
users  to  correctly  use  available  instrumentation  and  standards  to  perform  their  needed 
measurements  at  the  required  accuracy.   EEEL  metrology  research  is  focused  on  both 
components. 

The  primary  clientele  EEEL  serves  is  the  electronics  industry,  taken  very  broadly    The 
measurement  capability  provided  by  EEEL  programs  supports  entities  ranging  from  giant  electric 
utilities  and  large  aerospace  companies  to  specialized  instrument  manufacturers  employing  just  a 
few  individuals  EEEL's  electrical  measurements  help  advance  the  $350  billion  per  year  U.S 
electronics  sector.   For  example,  EEEL  provides  microwave  antenna  measurements  used  by 
every  major  U.S  aerospace  company,  thus  supporting  a  U.S  microwave  industry  which  supplies 
about  one  half  of  the  $60  billion  worid  microwave  equipment  market.   EEEL  provides  national 
reference  standards  that  are  needed  to  assure  the  accuracy  of  the  electric  power  meters  in  every 
home  and  business  in  the  U.S.,  thus  supporting  the  metering  of  neariy  $200  billion  per  year  of 
electricity.   EEEL  provides  measurement  capability  reflected  in  voluntary  industry  standards 
affecting  the  manufacture  of  every  new  U.S. -made  optical  fiber  communications  line,  thus 
supporting  a  US  fiber  optics  industny  supplying  a  domestic  market  approaching  $3  billion  and 
competing  in  a  world  martlet  in  excess  of  $5  billion. 

EEEL's  estimated  FY97  staffing  level  is  283  full-time-equivalent  staff,  funded  through  direct 
congressional  appropnations,  transferred  funds  from  other  govemment  agencies  and  industry, 
and  fees  received  for  calibration  services.   EEEL  provides  the  largest  share,  over  40  percent,  of 
all  NISTs  calibration  services,  representing  $2  million  in  calibration  fee  income. 

Why  NIST/EEEL? 

NIST's  role  is  to  remove  measurement  barriers  that  require  NIST's  special  characteristics: 

an  impartial  position,  recognized  measurement  capability,  expertise  in  a  wide  range  of 
measurement  areas,  direct  access  to  national  standards,  and  imprimatur  as  the  lead  US  agency 
for  measurements.  These  characteristics  pemnit  NIST  to  do  what  individual  companies  cannot 
do:  gain  access  to  industry  information  needed  to  resolve  industry-wide  measurement  problems, 
develop  measurement  solutions  and  national  reference  standards  that  benefit  the  industry  as  a 
whole  across  diverse  product  lines,  and  gain  acceptance  of  those  solutions  by  US  industry  and 
its  intemational  customers  so  that  US.  products  can  be  sold  efficiently  here  and  abroad. 

Typical  examples  where  EEEL  perfoms  this  unique  role  are  the  following: 

EEEL  is  worthing  with  the  Semiconductor  Industy  Association  (SIA)  on  the  National 
Technology  Roadmap  for  Semiconductors  to  identify  and  respond  to  key  measurement 
technology  needs  that  must  be  met  in  the  next  15  years  for  the  industry  to  manufacture  future 
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generations  of  electronic  chips.  The  SIA  has  written  a  strong  statement  of  the  need  for 
metrology,  and  further,  of  NIST's  role:  "NIST  is  the  only  place  in  the  U.S.  where  the  broad 
range  of  measurements  needed  for  semiconductor  processing  are  routinely  and 
systematically  developed."^ 


EEEL  develops  and  maintains  the  national  standards  for  laser  power  and  energy.  A  major 
instrument  manufacturer  has  reported  that  the  standards  from  EEEL  provide  the 
infrastructure  for  its  business  because  the  accurate  measurement  of  optical  power  is 
common  to  neariy  every  type  of  fiber  optic  test  instrument  being  manufactured.   NIST  is  the 
only  organization  recognized  as  capable  of  developing  unbiased  standards  as  it  has  the 
technical  expertise  required  for  developing  such  standards  and  has  no  financial  stake  in  the 
industry. 

New  infonmafion  technologies  increasingly  incorporate  wireless  communication  systems  and 
displays.   EEEL  is  developing  and  maintaining  the  national  impedance  and  microwave  power 
measurement  standards  which  underpin  the  measurements  needed  for  wireless 
communication.  Similariy,  EEEL  is  developing  test  procedures  needed  to  permit  international 
commerce  in  displays.  These  procedures  provide  unbiased  comparative  measures  for  U.S. 
manufacturers  who  purchase  displays  for  incorporation  into  their  systems.   In  both  these 
cases  the  international  reputation  of  NIST  for  impartiality  and  accuracy  is  required  to 
attain  the  recognition  needed  to  support  international  trade 

Determining  Industry  and  Government  Needs 

The  specific  client  needs  addressed  by  EEEL  are  driven  by  the  fast  moving  worid  of 
electronics — an  industry  that  continues  to  outstrip  its  capability  to  measure  and  control  its 
materials  and  processes  and  evaluate  its  products..  There  is  also  a  broad  new  recognition  in 
industry  of  the  significance  of  accurate  physical  standards  and  of  the  processing  and 
performance  standards  that  depend  on  them.  This  comes  about  as  industry  is  forced  to  meet 
conformity  requirements  in  the  intemational  arena  and  cost  and  quality  requirements  in  its 
manufacturing  plants.  All  of  these  pressures  demand  advances  in  measurements. 

EEEL  depends  on  continued  close  interaction  with  the  industry  to  help  determine  client  needs 
and  plan  our  response.   For  example,  just  in  the  past  two  years  we  have  sponsored  or  co- 
sponsored  nine  major  workshops  (see  Appendix  A)  and  participated  in  several  national  technical 
roadmapping  activities — which  are  particulariy  useful  in  identifying  industry  needs.   EEEL  staff 


^Semiconductor  Technology— Workshop  Conclusions,  Semiconductor  Industry  Association  (November  17- 
19,  1992) 
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also  actively  participate  in  conferences,  make  visits  to  industry  laboratories,  hold  telephone 
consultations,  conduct  and  attend  numerous  meetings.  Altogether,  we've  collaborated  with,  or 
served  in  some  substantial  way,  over  2000  different  organizations  in  the  past  two  years.  We 
typically  receive  useful  programmatic  guidance  and  feedback  from  these  collaborations. 

EEEL  Program  Selection 

In  the  face  of  the  large  range  of  needs  and  our  limited  resources,  we  must  choose  and  plan  our 
projects  very  carefully  to  maximize  their  impact.  We  have  long-established,  written  criteria  to 
help  in  our  project  selection;  these  are  summarized  in  Appendix  B.  Additional  information  about 
our  program  planning  process  is  published  yeariy  in  our  program  plan  which  includes  a  summary 
of  all  the  current-year  activity  in  the  Laboratory,  and  for  each  technical  area,  specific 
accomplishments,  a  description  of  the  significance  to  industry  and  govemment,  quantitative 
milestones  for  the  next  year,  and  an  updated  five-year  plan  giving  a  longer  term  view 

EEEL  also  conducts  selected  area-specific,  detailed  studies  which  provide  part  of  the  basis  for 
our  plans  and  budget  initiatives.   Nineteen  examples  of  significant  studies  (some  unpublished) 
are  included  in  Appendix  C.  A  current  example  is  in  our  program  on  electromagnetic 
compatibility  (EMC)  which  is  a  critical  issue  in  the  industry.   First  we  conducted  an  impact  study 
showing  a  high  retum  on  our  present  work,  but  nevertheless  we  reviewed  the  current  program  to 
see  if  our  impact  could  be  improved.  Next  we  analyzed  commercial  and  govemment  EMC 
standards  to  disclose  measurement  need,  and  then  we  conducted  a  field  study  involving  37 
organizations  and  90  people  to  carefully  define  specific  measurement  requirements.  And  finally, 
to  refine  the  priorities  of  a  very  broad  range  of  requests  for  assistance,  we  held  a  follow-on 
workshop  and  have  now  reoriented  our  program  to  concentrate  on  the  two  highest  priority  topics. 

EEEL  also  develops  substantial  technical-area  reviews  that  summarize  major  industrial 
measurement  requirements.   In  recent  years,  they  have  been  published  in  a  document  "Emerging 
Technologies  in  Electronics...  and  their  Measurement  Needs''^  and  more  recently  "Measurements 
for  Competitiveness  in  Electronics."^  These  have  both  received  industrial  review  at  levels  from 
management  to  the  bench  to  assure  their  relevance  and  validity.  Although  they  were  developed 
for  our  internal  planning,  we  have  already  received  over  900  requests  for  copies  of 
"Measurements  for  Competitiveness  in  Electronics."  Our  clients  have  told  us  they  find  it  useful  for 
their  own  intemal  planning. 

Evidence  of  Impact  of  EEEL  Programs 


'■  Emerging  Technologies  in  Electronics...  and  ttieir  Measurement  Needs,  NISTIR  90-4260  (February  1990). 
'  Measurements  for  Competitiveness  in  Electronics,  NISTIR  4583  (April  1993). 
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There  is  considerable  evidence  that  our  priority-setting  process  and  laboratory  efforts  work  well. 
Over  a  dozen  formal  econometric  studies  conducted  by  external  experts  have  shown  our  woric 
has  paid  off.   For  example,  our  work  in  electromagnetic  interference  saved  automotive 
manufacturers  alone  more  than  $16  million  a  year  and,  overall,  resulted  in  a  social  rate  of 
return  on  investment  of  266  percent.''  Our  optical  fiber  work  yielded  423  percent.^  In  studies  in 
four  semiconductor  areas  the  retum  varied  from  63  to  181  percent.^  As  our  program  resources 
are  small  compared  to  the  overall  needs  we  can  efficiently  target  high  impact  areas,  thus  our 
median  value  of  social  rate  of  retum  on  investment  is  well  over  twice  the  56  to  99  percent  retum 
found  for  typical  private  sector  innovation.'  (The  economic  experts  that  helped  us  in  these 
studies  assure  us  that  ours  are  conservative  figures  since  they  are  typically  based  on  just  a 
single  step  in  the  economic  commercialization  process  and  do  not  include  many  downstream 
benefits  which  themselves  are  very  significant.) 

We  constantly  inquire  of  our  clients  about  their  use  of  our  work  and  collect  evidence  of  our 
impact.  The  Director  of  the  Instruments  Division  of  the  Aubum  Group  of  Coherent  (a  U.S.  laser 
manufacturer)  says  the  NIST  approach  for  laser  profiling  measurements  "will  provide  the  laser 
industry  with  standards  in  an  important  area  that  has  continued  to  evade  us  for  almost  30 
years."  The  President  of  Rotek  instrument  Corporation  (a  small  U.S  electrical  instrument 
manufacturer)  has  told  us,  "The  technical  assistance  you  provide  throughout  the  year  greatly 
assists  us  in  maintaining  our  standards  and  enables  us  to  provide  the  best  service  possible  to 
our  customers."  The  Chief  Engineer  of  the  Motorola  Electromagnetics  Laboratory  said,  "  ..the 
technical  contributions  of  NIST  to  the  industry  have  been  outstanding  throughout  the  years."  The 
President  of  Optical  E.T.C.,  Inc.  had  this  to  say:  "As  a  small  business,  OETC  could  not  have 
reasonably  developed  the  TFPD  [thermal  flat  panel  display]  technology  without  the  collaboration 
of  NIST  personnel  and  access  to  the  extensive  facilities  at  NIST..." 

The  bottom  line  is  that  through  its  focus  on  measurement  research  and  services,  EEEL  really 
makes  a  difference  to  the  electronics  industry.  Our  metrology  activities  are  unique,  are  badly 


'  Albert  N.  Unk,  Estimates  of  Economic  Impact  of  NtST  Research  in  Electromagnetic 

CompatibilityAnterference  (Eft/IC/EMI)  fi^etrology.  p.  12-18  and  20  (December  1991). 

'  Albert  N.  Link,  Economic  Impact  of  NIST-Supported  Standards  for  ttie  U.S.  Optical  Fiber  Industry:  1981  - 

Present,  p.  i-ii,  16-17,  and  35-39  (February  1992). 

°  Charles  River  Associates  Incorporated,  Productivity  Impacts  ofNBS  R&D:  A  Case  Study  of  ttie 

Semiconductor  Technology  Program  Summary  Volume,  pp.  3, 19  and  20  (June  1981).  Albert  N.  Link, 

Economic  Impact  on  the  U.S.  Semiconductor  Industry  of  NIST  Research  in  Electromigration,  pp.  ii,  1  and 

30-39  (January  1992). 

'  Edwin  Mansfield,  John  Rapoport,  Anthony  Romeo,  Samuel  Wagner,  and  George  Beardsley,  "Social  and 

Private  Rates  of  Return  from  Industrial  Innovations",  Quarterly  Journal  of  Economics,  Vol.  91 ,  p.  233  (May 

1977).  J.G.  Tewksbury.  M.S.  Crandall,  and  W.  E.  Crane,  "Measuring  the  Societal  Benefits  of  Innovation". 

Science.  Vol.  209,  p.  659,  (August  8,  1993). 
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needed  by  our  sophisticated  industry,  and  have  a  major  impact  in  improving  productivity 
and  profitability. 

A  few  examples  serve  to  illustrate  the  breadth  and  impact  of  our  technical  expertise. 

Watt-hour  meters  are  used  to  meter  nearly  $200  billion  worth  of  electricity  annually  in  the 
U.S.  The  meter  manufacturers,  the  electric  utilities,  and  State  regulators  perform  about 
8,000,000  calibrations  of  these  meters  per  year   EEEL  performs  about  100  calibrations  per 
year,  which  are  sufficient  to  maintain  the  consistency  of  the  system  across  the  Nation.  Our 
research  in  support  of  electric  utilities  power  and  energy  calibrations  shows  that  our 
laboratory  research  has  a  social  rate  of  return  of  more  than  400  percent^ 

Intemational  trade,  particularly  in  high  technology  products,  depends  on  accurate 
measurements    In  the  electrical  area,  for  example,  EEEL  is  continually  involved  in  numerous 
intemational  comparisons,  conducted  between  national  measurement  laboratories  under  the 
auspices  of  the  Intemational  Bureau  of  Weights  and  Measures,  to  ensure  the  equality  of 
electrical  units  woridwide    These  comparisons  are  a  factor  in  preventing  the  creation  of 
non-tariff  trade  barriers  requiring  US.  instrumentation  and  components  to  be  re-certified  in 
the  country  of  import,  resulting  in  extra  delay  and  cost.   In  extraordinary  cases,  however, 
EEEL  has  been  asked  to  help  resolve  specific  problems,  including  the  export  of  transfonmers 
to  an  African  country,  the  export  of  electronic  instrumentation  to  Europe,  and  the  export  of 
watt-hour  meters  to  China.  The  companies  in  the  other  countries  suspected  the  accuracy  of 
tests  conducted  by  private  US.  companies  until  it  was  demonstrated  that  the  measurements 
were  consistent  with  the  official  measurement  systems  developed  and  maintained  by  EEEL. 

EEEL  introduced  the  theory  and  methods  that  pennit  indoor,  near-field  scanned 
measurements  of  antennas  to  replace  costly  and  low-resolution  outdoor  measurements,  and 
further,  to  allow  diagnostics  and  adjustments  to  be  made  in  phased  array  antennas  during 
actual  manufacture    As  a  result  some  15  U.S.  companies  have  installed  40  near-field 
calibration  facilities.  One  company  has  reported  a  $35  million  saving  and  another  a 
reduction  in  testing  time  from  a  month  to  12  hours  using  such  facilities. 

The  EEEL  microwave  program  has  introduced  new  measurement  methods  based  on 
transmission  line  test  structures  formed  directly  on  the  semiconductor  wafers.  This  work 
disclosed  large  errors  (greater  than  30%  in  some  cases)  in  traditional  industrial 
measurements  that  previously  frustrated  design  and  reduced  productivity.  These  results 


'  Albert  N.  Link,  An  Evaluation  of  the  Economic  Impacts  Associated  with  the  NIST  Power  and  Energy 
Calibration  Services.  Electronics  and  Electrical  Engineering  Latwratory,  National  Institute  of  Standards  and 
Technology,  Report  No.  NISTIR  5565  (January  1995). 
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have  been  incorporated  in  commercial  test  instruments  and  adopted  by  major  US. 
companies.   Eminent  people  in  the  field  have  praised  this  work,  one  calling  it  "the  greatest 
advance  in  microwave  circuit  theory  in  50  years". 

Our  most  basic  measurement  research  has  to  advance  rapidly  to  keep  abreast  or  ahead  of 
this  very  fast  moving  industry's  practical  needs.   For  example,  as  one  result  of  EEEL 
research  in  superconductivity,  we  discovered  how  to  make  operable  a  large  array  of 
Josephson  junctions.  Those  arrays  now  serve  as  the  national  standard  for  the  volt,  not  only 
in  this  country  but  in  many  others  as  well.  And  our  development  of  a  still  larger  array,  with 
over  20,000  junctions  providing  a  ten-volt  output,  tumed  out  to  be  essential  for  Hewlett- 
Packard  to  be  able  to  introduce  the  world's  then  most  advanced  and  accurate  multimeter.   H- 
P  and  other  manufacturers  use  the  array  for  manufacturing  and  quality  control,  and  small 
companies  have  commercialized  the  NIST  instrumentation  that  operates  the  arrays.  So,  now 
the  best  physical  standard  for  voltage  in  the  world  is  at  work  every  day  in 
manufacturing  plants  and  EEEL  faces  a  new  challenge  in  meeting  the  needs  of  the  future. 

As  part  of  its  wori<  to  improve  measurements  for  power  transistors,  EEEL  has  developed  an 
electro-thenmal  model  for  the  insulated-gate  bipolar  transistor.  This  has  led  to  a  revolutionary 
concept  that  permits  system  manufacturers  to  simulate  computationally  the  device 
performance  in  their  system,  and  to  make  a  purchase  decision,  before  the  device  is 
fabricated     This  research  led  to  an  effort  by  Motorola  and  Analogy,  Inc.  that  according  to 
Electronic  Engineering  Times,  "has  customers  getting  'virtual  prototypes'. ..in  lieu  of  actual 
samples    .    Using  simulation  models  will  typically  shave  five  months  off  the 
development  cycle,  which  for  power  systems  often  ains  about  a  year...  This  is  a  new 
paradigm  in  power  electronics."  The  model  has  also  been  adopted  by  the  Ford  Motor 
Company  in  the  design  of  electronic  systems  for  its  products. 

Over  the  years,  EEEL  work  with  the  optical  fiber  industry  has  provided  the  technical  basis  for 
over  two  dozen  Telecommunications  Industry  Assoaation  (TIA)  standards,  leading  the 
President  of  the  TIA  to  say,  "Without  the  NIST  assistance  and  leadership,  the  U.S.  fiber 
optics  industry  would  not  be  in  the  competitive  position  it  is  today."  More  recently  it  was 
found  that  U.S.  manufacturers  were  not  competitive  with  overseas  manufacturers  who  could 
make  optical  fibers  with  the  circular  cross-section  with  sub-micrometer  tolerances  needed  to 
avoid  large  signal,  losses  in  multiple  connections.  Responding  to  a  TIA  request,  EEEL 
provided  U.S.  manufacturers  with  improved  measurement  methods  and  a  standard  reference 
material  (SRM)  that  solved  their  problem,  and  now  all  the  fiber-drawing  towers  of  the  major 
U.S.  manufacturers  use  these  SRMs  for  production  control. 

Software-based  electronic  instruments,  typically  using  analog-to-digital  converters,  require 
thousands  of  measurements  for  evaluation  and  demand  a  new  test  strategy  for  cost  effective 
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use  in  manufactunng  and  evaluation.   EEEL  developed  an  analysis  method  that  permits  a 
greatly  reduced  data  sample  to  give  accuracy  effectively  equivalent  to  a  full  test.   For 
example,  the  8192  measurements  for  a  13-bit  converter  can  be  replaced  by  only 
64  measurements,  reducing  test  time  90  percent,  to  2  seconds.  One  manufacturer  expects 
that  using  this  improved  method  will  reduce  their  test  cost  from  $8  to  $3  per  $25  converter, 
translating  to  an  annual  saving  of  $400,000. 

•      In  recent  work  in  magnetics,  EEEL  developed  measurement  methods  to  study  nanoscale 
friction  forces  on  recording  disks  in  a  collaboration  with  a  major  manufacturer  of  computer 
disk  drives.  The  forces  arise  from  sticky  projections  20  or  30  nanometers  high  on  a  hard  disk 
platter  which  interfere  with  the  reduced  flying-height  of  the  magnetic-signal  reading  heads 
needed  for  Increased  storage  density.   EEEL  showed  how  atomic  force  microscopy  could  be 
adapted  to  measure  the  lateral  forces  involved  thus  providing  the  industry  with  a  tool  with 
which  to  identify  the  sources  of  increased  friction. 

The  National  Semiconductor  Metrology  Program 

A  topic  of  emphasis  in  this  hearing  is  NIST's  National  Semiconductor  Metrology  Program  which  is 
a  NIST-wide  program  managed  by  EEEL    The  semiconductor  industry's  measurement  needs  are 
constantly  escalating,  being  driven  by  the  increasing  complexity  of  integrated  circuits  and  the 
requirements  for  tighter  control  of  more  numerous  and  more  sophisticated  manufacturing 
processes.   Here  the  industry  is  defined  broadly  to  include  integrated  circuit  manufacturers, 
materials  suppliers,  and  makers  of  semiconductor  manufacturing  equipment.  Three  important 
reasons  lead  NIST  to  work  intensively  to  meet  the  measurement  needs  of  the  semiconductor 
industry 

-  First,  that  industry  is  critical  to  the  success  of  the  electronics  industry    And  electronics 
in  tum  is  a  critical  element  in  the  modemization  and  success  of  very  neariy  every  segment  of 
our  economy  including  automobiles,  banking,  manufacturing,  communication,  defense, 
entertainment,  computers,  and  transportation. 

-  Second,  making  semiconductors  is  an  extraordinarily  measurement  intensive 
undertaking.  A  wide  range  of  specialized  scientific  and  engineering  skills  Is  required  to 
develop  and  improve  the  measurement  understanding  and  application  techniques  that  are 
needed.  The  Semiconductor  Industry  Association  made  this  statement  about  the  importance 
of  metrology  to  its  industrial  processes:  "Metrology  is  a  key  enabler  for  equipment  design  and 
manufacture,  for  materials  production,  for  generating  data  for  modeling  throughout 
semiconductor  manufacturing,  and  for  final  test  and  evaluation.  Without  satisfactory 
metrology,  the  industry  .    will  cease  to  be  competitive." 
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-    Third,  the  Semiconductor  Industry  Association's  National  Technology  Roadmap  for 
Semiconductors  provided  clear  input  from  industry  on  what  technologies  would  be 
needed  and  when. 

Early  EEEL  Contributions 

EEEL  has  a  40  year  history  in  supplying  nneasurement  technology  to  the  semiconducor  industry. 
These  contributions  began  with  two  small  projects  that  have  benefitted  the  industry  by  more  than 
$30  million  (more  than  100  times  the  cost  of  the  work)  in  marketplace  exchanges  alone.  These 
projects,  and  those  that  have  followed,  have  been  selected  by  working  with  the  semiconductor 
manufacturers  and  equipment  suppliers  to  understand  their  needs  for  measurements  and  to 
match  the  most  important  of  those  needs  with  EEEL's  available  resources.  The  products  of 
this  interactive  process  are  relevant  to  real  needs,  workable,  and  well  documented,  as 
typically  demonstrated  by  interiaboratory  test.  Many  are  delivered  via  physical  Standard 
Reference  Materials  for  direct  calibration  of  measurements  in  the  client's  plant.  Often  significant 
training  is  available  to  assure  proper  application  of  the  measurement  technique.  And,  as 
indicated  eariier,  EEEL  has  studied  the  economic  impacts  of  its  semiconductor  work  to 
understand  its  value  in  those  terms. 

Some  example  EEEL  contributions  are: 

•  Industry  recognized  that  unpredictable  wire  bond  failures  were  a  major  system  reliability 
problem  but  did  not  know  why  the  failures  were  occurring.  As  EEEL  staff  developed  the 
measurement  methods  destined  to  solve  the  problem,  they  discovered  many  factors 
underiying  the  failures.  This  work  established  a  new  metallurgical  understanding  and  new 
procedures  that  led  to  radical  changes  in  the  design  of  commercially  available 
equipment.   It  also  increased  productivity  in  high  reliability  circuit  manufacture  by  as  much  as 
35  times. 

•  EEEL  work  on  photomask  linewidths  led  to  the  reduction  of  intercompany  measurement 
discrepancies  by  an  order  of  magnitude,  stimulated  the  production  of  new  commercial 
instrumentation,  extended  the  range  of  use  of  optical  microscopes,  and  produced  a  net 
saving  of  $177  per  $1000  mask  for  a  $30  million  saving  to  mask  manufacturers  alone, 
according  to  a  1982  study.    The  provided  techniques  and  calibration  standards  have  been 
adopted  industry  wide.  Hewlett-Packard,  IBM,  and  Texas  Instnjments,  for  example,  ceased 
use  of  their  different  intemal  standards  and  adopted  those  of  NIST. 

Developing  reliable  mathematical  models  is  an  essential  part  of  EEEL  metrology  research. 
Electromigration  failures  occur  at  high  current  densities  when  the  electron  flow  in  the 
semiconductor  device  actually  moves  material  and  breaks  a  connection.  Our 
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electromigration  results  have  been  used  by  at  least  14  companies,  and  an  external  study 
has  estimated  industrial  savings  of  $26  million  in  return  for  the  $1  6  million  spent  on  the 
project.   Further,  the  results  are  the  basis  for  three  industrial  standards,  and  one  U.S. 
company,  Sienna  Technologies,  Inc.,  supplies  test  stations  and  a  library  of  standard 
structures  based  on  EEEL's  recommendations. 

The  NSMP 

The  National  Semiconductor  Metrology  Program  draws  on  the  breadth  of  expertise  in  the  NiST 
Laboratories  as  well  as  having  collaborative  relations  with  a  wide  range  of  organizations  including 
Sandia  National  Laboratories,  Lincoln  Laboratories,  Semiconductor  Research  Corporation,  and 
SEMATECH,  in  addition  to  those  already  existing  with  the  industry. 

Internally,  program  selection  has  been  aided  substantially  by  the  development  of  the  National 
Technology  Roadmap  for  Semiconductors,  which  identifies  in  detail  the  technical  needs  of  the 
industry  projected  over  the  next  15  years.  The  Roadmap  and  its  predecessors  have  repeatedly 
identified  metrology,  the  science  of  measurement,  as  one  of  a  very  few  "crosscutting"  issues,  i.e., 
issues  that  affect  every  area  of  semiconductor  design,  development,  and  manufacturing. 
With  initial  funding  of  $4.8  million,  a  current  funding  of  about  $10  million  and  an  annual  funding 
objective  of  $25  million,  the  work  is  focused  on  the  mainstream  technology  addressed  by  the 
Roadmap,  digital  silicon  integrated  circuits. 

The  NSMP  supports  measurement  development  programs  in  silicon  semiconductor  materials; 
dimensional  measurements  for  patterns  on  the  surface  of  silicon  wafers;  trace  moisture  in 
process  gases;  physical  and  chemical  properties  data  for  modeling  chemical  reactions;  pressure, 
composition,  and  flow  rate  of  process  gases;  boron  implants  in  silicon;  surface  microroughness 
of  silicon  wafers;  improved  high-temperature  thermocouples;  properties  of  polymers  for  device 
packages;  and  intensity  of  ultraviolet  light  for  making  fine  pattems  on  wafer  surfaces. 

Among  the  results  to  date  have  been  Standard  Reference  Materials  for  oxygen  and  boron  in 
silicon,  new  in-house  designed  and  constructed  equipment  for  absolute  moisture  measurements 
in  gases  to  extremely  low  levels,  electrical  measurements  of  alignment  between  successive 
layers  of  materials  with  accuracy  of  less  than  30  atom  spaces,  calibration  services  for  ultraviolet 
lightmeters,  greatly  improved  understanding  of  the  chemistry  of  electrically  driven  film  deposition 
processes,  and  better  knowledge  conceming  the  scattering  of  light  from  highly  polished  surfaces 
that  affects  the  detection  of  small  particles  of  contaminating  materials  on  those  surfaces.  These 
contributions  are  too  new  to  permit  full  evaluation  of  their  technical  or  economic  impact,  but  as 
an  example,  the  last  of  these  has  already  solved  a  major  problem  for  a  leading  U.S. 
manufacturer  of  semiconductor  manufacturing  equipment. 
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Funding  increases  for  NSMP  requested  through  the  Budget  Initiative  process  for  FY97  will  be 
used  to  address  metrology  issues  associated  with  193  nanometer  optical  lithography  Recent 
changes  in  the  missions  (and  funding  policies)  of  both  DARPA  and  SEMATECH  coupled  with 
fundamental  business  changes  for  larger  firms  like  IBM  Microelectronics  and  AT&T/Lucent 
Technologies  have  left  a  void  in  this  chtical  path  development  which  NIST  is  working  to  fill, 
primarily  through  active  partnerships  with  other  organizations.  The  end  goal  is  to  enable  at  least 
one  domestic  supplier  to  produce  a  viable  advanced  optical  lithography  tool  to  meet  the  180 
to  130  nanometer  technology  needs  on  the  schedule  of  the  National  Technolgoy  Roadmap  for 
Semiconductors  so  that  US.  industry  will  not  be  constrained  to  buying  next  generation 
semiconductor  manufacturing  tools  from  foreign  sources. 

I  would  like  to  conclude  with  five  fundamental  points  about  our  research  programs.  These 
programs  are  laboratory  based,  and  need  to  be,  support  the  globalization  of  trade,  are  a  high- 
leverage  activity,  are  unique  in  the  United  States,  and  are  coordinated  with  industry.   Thank  you 
for  your  attention  and  I  will  be  pleased  to  answer  any  questions  you  may  have. 


12 


84 

Appendix  A 

EEEL  Workshops 
(March  1994 -April  1996) 

1.  International  Workshop  on  Semiconductor  Characterization:  Present  Status  and 
Future  Needs 

2.  Testing  for  Wireless  Applications 

3.  Compound  Semiconductor  Technology 
(NIST/SEMI) 

4.  Measurement  and  Characterization  of  Ultra-Shallow  Doping  Profiles  in 
Semiconductors 

(NIST/American  Vacuum  Society/lntel/Microelectronics  Center  of  North 
Carolina/SEMATECH) 

5.  EMI/EMC  Metrology  Challenges  for  Industry:  A  Workshop  on  Measurements 
Standards  Calibration  and  Accreditation 

6.  Display  Materials  Consortium  Workshop 

7.  Antenna  Parameter  Measurement  by  Near-Field  Techniques 

8.  Testing  Strategies  for  Analog  and  Mixed-Signal  Products 

9.  Advanced  Digital  Video  in  the  National  Information  Infrastructure 
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Appendix  B 

Summary  of  EEEL  Program  Selection  Criteria 

1.  FIT  TO  MISSION 

Goals  Served  -  Primary  goal  is  to  strengthen  the  U.S.  economy  via  contributions  to 

improving  the  competitiveness  of  U.S.  industry. 
Customers  Served  -    Industry  primarily,  then  Government,  then  educational  institutions. 

Deliverables  -  Measurement  accuracy  and  capat^ility  are  highest  priority  deliverables. 

Steps  Served  -  Service  to  marketplace  exchange  Is  primary,  service  to  manufacturing  a 

close  second. 
Uniqueness-  Only  NISI  can  and  v^ll. 

2.  IMPACT  IF  SUCCESSFUL 

Market  Size  -  Work  supporting  a  large  market  (direct  and  indirect)  higher  priority  than  a 

small  market. 
Market  Immediacy  -     Work  supporting  a  market  that  currently  exists  gets  a  higher  priority  than 

work  supporting  a  market  that  won't  exist  until  some  time  in  the  future. 
Timeliness  -  Relative  to  the  time  of  need,  work  that  will  be  delivered  when  first  needed 

gets  a  higher  priority. 
Criticality  -  The  "stopping  power"  of  the  lack  of  the  NIST  contribution.  A  product 

cannot  be  made,  or  marketed,  or  supported  without  the  NIST  work. 
Sensitivity  -  Willingness  of  industry  client  to  provide  support  for  the  work. 

Visibility  -  The  more  visible  the  impact  of  the  proposed  work,  the  higher  the  priority. 

Commitment  -  How  efficiently  staff  time  is  used;  one-time  ,  periodic,  or  continuous. 


3.  PROBABIUTY  OF  SUCCESS 

In  completing  technical  work  and  achieving  adoption  by  client;  based  on 
adequacy  of  staff,  equipment,  and  funding. 

4.  INSTITUTIONAL  HEALTH 

Projects  that  strengthen  the  organization  and  staff,  promote  growrth  and 
recognition. 
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Appendix  C 

Examples  of  Special  EEEL  Planning  Studies 

Silicon  resistivity 

Wirebonding 

Photomask  iinewidth 

Magnetics  calibration 

Magnetics  program 

Center  (Laboratory)-wide  calibration  services  policy 

Dielectric  materials 

Low-frequency  electronics 

Lightwave  metrology 

Microwave  metrology 

National  laboratories  calibration  service  benchmarking 

Advanced  semiconductor  metrology 

Lasers 

Optical  fiber  metrology 

Optical  fiber  sensors 

Electromagnetic  compatibility 

Video 

Strategic  planning  studies 

Fields  and  interference  metrology 
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Robert  E.  Hebiier 

Acting  Deputy  Director 

National  Institute  of  Standards  &  Technology 

Robert  E.  Hebner  received  his  Ph.D.  in  physics  from  the  University  of  Missouri  in  1971. 
He  joined  what  was  then  the  National  Bureau  of  Standards  as  a  presidential  intern.    Since 
that  time  he  has  advanced  through  increasingly  responsible  positions  and  is  currently  Acting 
Deputy  Director  of  NIST. 

His  activities  with  other  agencies  have  included  assignments  at  the  Office  of  Management  and 
Budget,  at  Sandia  National  Laboratories,  and  most  recently,  at  the  Advanced  Research 
Projects  Agency  of  the  Department  of  Defense.    He  has  served  as  a  member  of  the  Advisory 
Committees  for  two  NATO  Advanced  Study  Institutes  and  has  participated  in  the 
organization  of  many  national  and  international  conferences.    Dr.  Hebner  was  a  member  of 
the  review  committee  for  U.S.  Department  of  Energy's  Office  of  Energy  Storage  and 
Conservation  in  1986  and  on  an  NSF  Panel  for  Engineering  Research  Equipment  Grants.    In 
addition,  he  served  on  the  National  Research  Council's  Committee  to  Conduct  an  Assessment 
on  Establishing  Underwater  Arcs  and  Flames.    He  is  a  member  of  a  Federal  interagency 
committee  to  coordinate  the  U.S.  research  and  communication  program  on  the  potential 
health  effects  of  electric  and  magnetic  fields.    He  was  the  principal  investigator  and  the 
government's  key  witness  in  a  celebrated  court  case  involving  an  alleged  novel  approach  to 
the  efficient  generation  of  electricity. 

Throughout  his  career.  Dr.  Hebner  has  been  active  in  technical  activities  having  authored  or 
coauthored  more  than  fifty  technical  papers  and  reports.    He  is  the  past  president  of  the 
Dielectrics  and  Electrical  Insulation  Society  of  the  Institute  of  Electrical  and  Electronics 
Engineers.    In  addition,  he  has  served  on  numerous  technical  committees  which  develop 
voluntary  standards  for  the  electric  utility  industry. 

Dr.  Hebner  is  a  member  of  the  American  Association  for  the  Advancement  of  Science,  the 
American  Physical  Society,  and  is  a  fellow  of  the  institute  of  Electrical  and  Electronics 
Engineers. 

The  awards  that  he  has  received  include  the  U.S.  Department  of  Commerce  Silver  and 
Bronze  Medals  and  the  1990  Harry  Diamond  Memorial  Award  given  annually  by  the  Institute 
of  Electrical  and  Electronic  Engineers  for  outstanding  technical  contributions  in  the  field  of 
government  service.    The  award  cites  Dr.  Hebner's  contribution  to  the  development  of 
electrical  measurements  and  standards  for  the  electrical  utility  industry. 
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Mr.  GUTKNECHT.  Thank  you,  Dr.  Hebner. 

I  think  before  we  take  any  questions  we  will  go  to  Dr.  Rhyne  and 
Dr.  Glaze  and  then  we  will  have  questions.  Dr.  Rhyne? 

STATEMENT  OF  DR.  V.  THOMAS  RHYNE 

Dr.  Rhyne.  Thank  you  very  much.  It's  my  fault,  or  actually,  it's 
the  fault  of  a  major  domestic  air  carrier  who  had  a  little  computer 
trouble  this  morning  that  I  forced  the  switch  and  I  appreciate  your 
being  flexible  and  I  apologize. 

I  want  to  also  thank  the  Members  of  the  Committee  for  inviting 
me  and  for  having  this  opportunity  to  comment  on  the  National  In- 
stitute of  Standards  and  Technology's  Electronics  and  Electrical 
Engineering  Laboratory,  and  my  focus  is  going  to  be  on  the  assess- 
ment process  which  provides  an  annual  measure  of  the  effective- 
ness of  this  key  government  program. 

As  my  name  tag  shows,  I'm  Tom  Rhyne.  I'm  the  current  Chair 
of  the  National  Research  Council's  Panel  on  Assessment  for  what 
we  commonly  call  EEEL.  When  you  look  at  Electronics  and  Elec- 
trical Engineering  Laboratory,  I  think  you  can  understand  why  we 
simplify  to  that. 

I've  been  involved  in  the  assessment  process  at  NIST  for  over 
five  years  and  I've  worked  in  national  and  international  standards 
bodies  for  over  20  years.  I  currently  work  within  the  semiconductor 
product  sector  at  Motorola.  Prior  to  that,  I  worked  at  the  Microelec- 
tronics and  Computer  Technology  Corporation,  which  you  probably 
better  know  as  MCC.  I've  also  spent  about  20  years  teaching  elec- 
trical engineering  and  computer  engineering  at  a  number  of  U.S. 
universities. 

Each  year,  the  work  of  the  panel  consists  primarily  of  performing 
an  assessment  of  the  technical  work  being  conducted  by  the  EEEL, 
the  general  nature  of  which  and  some  of  the  highlights  of  which 
have  already  been  described  to  you  by  Dr.  Hebner.  The  current 
panel  consists  of  some  20  senior  engineers  and  scientists  selected 
from  14  U.S.  corporations,  including  my  own  company,  Motorola, 
AT&T,  Bell  Laboratories,  Scientific-Atlanta,  Coming,  Hughes  Air- 
craft, American  Superconductor  Corporation,  and  Texas  Instru- 
ments, just  to  name  a  few.  We  have  members  from  two  national 
consortia  research  centers,  the  Semiconductor  Research  Corpora- 
tion in  North  Carolina  and  SEMATECH,  and  one  university. 

In  performing  the  annual  assessment,  each  of  these  men  and 
women  spends  as  much  as  a  week  away  from  their  normal  tech- 
nical and  managerial  duties  traveling  to  Washington  or  Colorado  to 
discuss  the  laboratory's  technical  activities  with  the  managers  and 
technical  staff  at  the  laboratories,  and  therefore,  the  committee, 
the  panel  works  together  to  prepare  an  extensive  written  report  on 
their  findings. 

All  of  this  work  is  performed  by  these  men  and  women  on  a  pure- 
ly pro  bono  basis,  a  significant  tribute  to  the  importance  that  the 
employers  of  these  people  and  the  companies  those  employers  rep- 
resent place  on  this  particular  government  laboratory.  In  fact,  in 
and  of  itself,  the  willingness  of  the  corporations  to  allow  us,  as 
panel  members,  to  spend  this  time  away  fi-om  our  jobs  is  a  major 
measure  of  the  high  regard  which  the  associated  United  States  in- 
dustries place  on  the  work  of  the  EEEL. 
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My  written  remarks,  which  I  believe  have  already  been  provided 
to  you  in  advance,  serve  to  summarize  the  most  recent  findings  in 
1995  of  the  Panel  on  Assessment.  They  touch  briefly  on  the  work 
being  performed  in  the  areas  within  which  the  EEEL  focuses,  in- 
cluding areas  such  as  electricity,  which  includes  both  basic  elec- 
trical standards  and  support  for  the  U.S.  electric  utility  industry, 
semiconductor  electronics,  electromagnetic  fields,  electromagnetic 
technology,  which  is  an  area  that  includes  superconductivity,  a 
very  important  field  in  the  current  time,  and  optoelectronics,  £ind 
also  the  National  Semiconductor  Metrology  Program.  All  of  these 
fields  are  obviously  of  major  importance  to  our  national  interests. 
Additional  detail  is  available  in  the  published  assessment,  and  if 
you  don't  have  a  copy,  I  am  sure  that  the  National  Research  Coun- 
cil will  be  glad  to  provide  you  one. 

In  my  brief  oral  remarks,  I  can't  really  cover  all  the  details,  but 
I  can  make  one  summarizing  statement;  NIST's  EEEL  is  doing  an 
excellent  job  in  meeting  as  much  of  the  metrology  and  standardiza- 
tions needs  of  the  industries  it  serves  as  its  current  resources  will 
allow. 

The  major  problem  that's  been  identified  by  the  panel  over  the 
last  several  years  is  the  mismatch  between  available  resources  and 
known  needs,  especially  in  the  last  few  years  as  what  NIST  calls 
OA,  or  other  agency  funding,  has  shifted  from  defense  agencies  and 
become  diminished.  That  shift  away  from  military  funding  has 
been  managed  very  well  by  this  part  of  the  NIST  operations,  but 
as  comme^-cial  needs  have,  expanded  and  as  the  costs  of  making  in- 
creasingly more  accurate  measurements  in  the  electrotechnical  do- 
main have  grown,  the  challenge  before  the  EEEL  has  increased 
and  they  now  face  and  will  continue  to  face  a  very  difficult  form 
of  what  I  call  technological  triage,  where  they,  as  the  managers  of 
the  laboratory,  must  select  which  projects  to  start,  which  projects 
to  maintain,  and  which  projects  to  stop. 

They  do  this  by  working  closely  with  those  industries  that  they 
serve  in  a  highly  symbiotic  relationship,  as  will  be  well  illustrated 
by  the  laboratory's  work  in  conjunction  with  the  National  Tech- 
nology Roadmap  for  Semiconductors,  which  Dr.  Glaze  will  describe 
to  you.  In  that  role,  the  EEEL  is  providing  much  needed  support 
to  a  very  important  sector  of  the  U.S.  industry,  and  without  such 
support,  the  global  industrial  stature  of  the  U.S.  semiconductor  in- 
dustry would  certainly  be  significantly  eroded. 

There  is  one  point  I  want  you  to  understand.  Standardization 
and  metrology  have  become  key  weapons  in  the  international  mar- 
ketplace, and  with  the  countries  of  the  European  Economic  Com- 
munity and  the  Pacific  Rim  often  voting  as  blocks  to  enforce  their 
preferred  national  standards  on  the  international  community,  the 
work  of  the  EEEL  in  proposing  new  electrotechnical  standards, 
new  metrology  methods,  and  in  helping  U.S.  industries  to  deploy 
rapidly  newly  adopted  international  standards  is  a  critical  enabler 
to  the  U.S.  global  industrial  competitiveness. 

Speaking  for  the  entire  panel,  we  commend  to  you  the  work  of 
the  EEEL  in  meeting  that  challenge.  Clearly,  this  is  one  area  of 
government  activity  which  provides  excellent  return  on  investment. 
Thank  you. 

[The  prepared  statement  of  Dr.  Rhyne  follows:] 
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Ms.  Chairwoman  and  Members  of  the  Subcommittee: 

I  am  pleased  and  privileged  to  address  you  today  as  the  Chair  of  the  National  Research 
Council's  Panel  for  Electronics  and  Electrical  Engineering,  one  of  seven  panels  of  the  Board  on 
Assessment  of  National  Institute  of  Standards  and  Technology  (NIST)  Programs    My  own 
training  is  in  electrical  engineering,  and  I  am  currently  the  Manager  of  Strategic  Programs  within 
the  Semiconductor  Products  Sector  of  Motorola,  Inc.  I  have  been  employed  in  the  electronics 
industry  for  over  thirty-five  years,  including  sixteen  years  as  a  Professor  of  Electrical  Engineering 
at  Texas  A&M  University.  I  have  served  for  five  years  on  the  Board  of  Directors  of  the  IEEE, 
and  have  been  Chair  of  the  Panel  for  Electronics  and  Electrical  Engineering  since  1993.  The 
Panel  currently  has  twenty  members  representing  a  variety  of  areas  of  expertise  in  electronics  and 
electrical  engineering. 

The  work  of  the  Panel  consists  primarily  of  performing  annual  assessments  of  NIST's 
Electronics  and  Electrical  Engineering  Laboratory,  more  simply  referred  to  as  the  EEEL,  the 
general  nature  of  which  has  been  described  to  you  by  its  director.  Dr.  Judson  French.  This  work 
is  purely  pro  bono,  with  the  Panel  members  taking  time  away  from  their  normal  jobs  to  spend  as 
much  as  a  week  each  year  visiting  the  Laboratory,  discussing  its  technical  activities  with  the 
Laboratory  managers  and  technical  staff,  and  preparing  an  extensive  written  report  on  their 
findings    The  current  panel  includes  employees  of  sixteen  different  U.S.  electronics  companies,  a 
tribute  to  the  importance  those  companies  assign  to  the  task  of  assessing  the  activities  of  this  key 
government  laboratory 

My  remarks  today  summarize  the  findings  of  the  Panel  during  its  1995  assessment. 

In  1995,  as  has  been  the  case  over  the  past  several  years,  the  Panel  found  the  programs  of 
the  EEEL  to  be  well  managed  and  of  high  quality.  The  selection  of  projects  within  the 
Laboratory  generally  appears  to  be  well  matched  to  the  current  needs  of  its  broad  range  of 
industrial  customers,  including  the  semiconductor  industry,  the  electric  power  utility  and 
equipment  industries,  developers  and  manufacturers  of  microwave  devices  and  other  wireless 
technologies,  and  the  magnetic  data  storage  industry    This  is  a  very  broad  charter,  however,  and 
given  the  rapid  pace  of  developments  in  the  technical  and  scientific  areas  associated  with  these 
industries,  coupled  with  the  limited  resources  available  for  supporting  its  work,  continual 
refinement  of  the  Laboratory's  priorities  is  a  major  challenge  to  its  managers    As  a  result,  the 
Laboratory  has  created  a  formal  prioritization  process  which  has  proven  to  be  largely  effective  in 
meeting  this  broad  challenge. 

During  its  assessment  activities,  the  Panel  has  identified  numerous  examples  of  work 
within  EEEL  which  has  resulted  in  measurable  impact  on  key  sectors  of  U.S.  industry.  For 
example,  a  NIST-developed  digital  impedance  bridge  has  allowed  the  first  significant 
improvement  in  inductance  measurements  in  40  years.  Based  on  this  advance  and  other  related 
NIST  improvements,  uncertainties  in  calibration  services  for  capacitance,  inductance,  and  loss  will 
soon  be  decreased  by  a  factor  of  10  or  more    The  Laboratory  has  also  refined  electrical  methods 
for  measuring  the  thickness  of  metal  films  and  improved  the  understanding  of  time-dependent 
dielectric  breakdown  in  thin  silicon-dioxide  films    These  and  other  accomplishments  are  providing 
much  needed  help  to  US  semiconductor  manufacturers  who  are  constantly  seeking  better 
control,  higher  accuracy,  and  higher  yields  within  their  fabrication  facilities. 
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The  EEEL  also  provides  standards  for  calibration  of  antennas  and  other  wireless 
components.  The  availability  of  these  standards,  which  have  known  error  bounds  and  are 
traceable  to  national  standards,  has  had  a  very  positive  impact  on  the  international 
competitiveness  of  the  US  telecommunications  industry.  The  Laboratory's  impact  can  also  be 
gauged  by  the  great  industry  demand  for  other  services  such  as  calibration  services,  by  the  level  of 
industry  participation  in  EEEL- sponsored  training  and  workshops,  and  by  the  increasing 
commercialization  of  EEEL  research  through  CRADAs  and  other  forms  of  government/industrial 
cooperation. 

Clearly,  the  impact  of  the  EEEL's  technical  and  scientific  work  has  been  most 
concentrated  in  sectors  in  which  industries  have  clearly  identified  their  needs  for  MIST  services. 
An  example  of  this  is  the  semiconductor  industry,  which  has  explicitly  stated  its  metrology  needs 
in  The  National  Technology  Roadmapfor  Semiconductors,  produced  by  the  Semiconductor 
Industry  Association,  a  major  industrial  effort  to  be  further  described  by  Dr.  Jim  Glaze  who  will 
follow  me.  Working  afield  fi^om  these  successes,  recent  Laboratory  efforts  to  assist  other  key 
industrial  sectors  in  identifying  their  present  and  future  needs  show  clear  promise  for  expanding 
the  role  of  the  EEEL  in  supporting  the  metrology  needs  of  those  sectors. 

As  I  previously  mentioned,  however,  the  EEEL  continually  faces  challenges  in  choosing 
priorities  in  its  program  planning    As  part  of  its  feedback  process,  the  Panel  has  identified  several 
areas  where  it  thinks  greater  attention  or  improvement  is  merited  in  current  and  near-term  EEEL 
programming.  For  example,  the  electric  power  industry  is  deregulating  in  a  manner  analogous  to 
that  of  the  telephone  industry,  and  metering  and  measurements  at  various  points  in  the  power 
system  wall  become  much  more  numerous  and  important  than  at  present    The  Laboratory's 
projects  related  to  electric  power  are  individually  valuable  and  of  high  quality,  however,  in  the 
view  of  the  Panel,  these  EEEL  efforts  are  having  only  minimal  impact  on  industry    The  Panel 
believes  that  the  Laboratory  is  in  a  position  to  make  important  contributions  if  it  can  achieve  the 
right  focus,  using  its  ties  to  industry  players  such  as  utility  companies,  independent  power 
producers,  and  electrical  equipment  manufacturers  to  develop  a  roadmap  of  metering  and 
measurement  needs  for  the  modem  electric  power  industry,  paralleling  the  efforts  of  the 
semiconductor  industry 

Similarly,  no  coherent  US  industrial  leadership  exists  in  the  development  of  international 
standards  for  commercial  superconductors,  and  the  Panel  has  recommended  that  NT  ST  assume 
national  leadership  in  this  area    Unfortunately,  the  present  level  of  NIST  staffing  and  support  in 
this  area  is  insufficient  to  compete  effectively  with  a  large  competing  Japanese  effort.  If  this  level 
is  not  supplemented  in  the  next  year,  the  Panel  has  expressed  strong  concern  that  the  result  may 
be  Japanese  domination  of  the  international  standards  in  this  critical  area. 

In  another  area  of  technology  of  major  national  importance,  it  has  been  clearly 
demonstrated  over  the  past  20  years  that  many  electrical  measurements  are  best  made  at  low 
temperatures    The  collective  impact  on  the  capabilities  of  NIST  in  the  metrology  and  standards 
arenas  has  been  vastly  enhanced  by  the  low-temperature  research  of  the  EEEL  at  both  its  Boulder 
and  Gaithersburg  campuses    Unfortunately,  the  current  budget  of  the  Boulder  group  appears  to 
be  frozen,  resulting  in  the  loss  of  one  staff  member  per  year    This  is  demoralizing  a  group  that  is  a 
unique  national  asset.  The  Panel  has  recommended  that  a  deliberate  decision  should  be  made  at 
the  Laboratory  level  to  increase,  maintain,  or  shrink  these  activities  to  an  appropriate  size,  rather 
than  letting  it  wither  away 
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As  American  industries  have  learned  over  the  past  several  decades,  quality  systems  are 
important  tools  for  improving  measurement  assurance,  competitiveness,  effectiveness  of  research 
and  development  activities,  and  the  quality  of  goods  and  services.  As  with  its  industrial 
counterparts,  the  Panel  has  recommended  that  a  documented  or  consistent  quality  system  would 
increase  the  value  of  the  Laboratory's  calibration  services  to  customers    The  Panel  has  also 
recommended  that  EEEL  should  prepare  for  the  effects  of  increasing  ISO  9000  registrations 
within  U.S.  companies    The  calibration  requirements  in  this  standards  series  are  more  stringent 
than  those  generally  practiced  by  US  industry,  and  increasing  ISO  9000  registrations  will  affect 
the  Laboratory's  workload  and  income  from  fees,  and  v»ill  also  require  continual  improvement  in 
these  services. 

The  Panel's  review  of  EEEL  staffing  has  found  some  turnover  in  projects  supporting 
fundamental  standards,  coupled  with  aging  of  the  Laboratory  staff  and  continued  dependence  on 
retired  staff  as  consultants.  The  maintenance  of  fundamental  standards  is  an  exacting  and  unusual 
specialty,  and  in  the  opinion  of  the  Panel,  the  Laboratory  needs  to  attract  qualified  young 
scientists  to  ensure  that  the  accumulated  knowledge  of  the  current  staff  is  transferred  to  them  as 
part  of  an  orderly  replacement  and  renewal  process 

Despite  facing  the  types  of  challenges  common  to  most  organizations  at  the  present  time, 
the  Electronics  and  Electrical  Engineering  Laboratory  is  overall  very  well  managed.  Its  staff  is 
dedicated,  and  represents  a  significant  resource  for  our  nation.  The  Laboratory's  technical 
programs  are  of  very  good  to  excellent  quality,  and  its  ties  to  its  customers  in  industry  are  strong. 
In  simple  terms,  the  Panel  has  repeatedly  found  the  EEEL  to  be  a  most  effective  national  asset 
which  is  playing  a  key  role  in  assuring  US  competitiveness  in  the  key  electrical  and  electronics 
industrial  sectors 
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Fiscal  Year  1995  Assessment  of  the 

Electronics  and  Electrical  Engineering  Laboratory 

by  the 

Panel  for  Electronics  and  Electrical  Engineering 

V  Thomas  Rhyne,  Motorola,  Inc  ,  Chair 

B.  Jayant  Baliga,  North  Carolina  State  University 

Gary  M  Davidson,  GM  Davidson,  Inc 

James  F  Freedman,  Semiconductor  Research  Corporation 

H  R.  Hofmann,  AT&T  Bell  Laboratories 

Roger  F  Hoyt,  IBM  Almaden  Research  Center 

James  D  Huff,  Scientific-Atlanta,  Inc 

Richard  I  Knight,  Tektronix,  Inc. 

Frederick  J  Leonberger,  United  Technologies  Photonics,  Inc. 

Suzanne  R  Nagel,  AT&T  Bell  Laboratories 

Arthur  A  Oliner,  Polytechnic  University 

Don  Parker,  Hughes  Aerospace  &  Electronics  Company 

Alton  D  Patton,  Texas  A&M  University 

D  Howard  Phillips,  Consultant,  Durham,  North  Carolina 

John  M.  Rowell,  Conductus,  Inc 

Robert  E  Schwall,  American  Superconductor  Corporation 

Thomas  J  Shaffher,  Texas  Instruments  Incorporated 

Horst  L  Stormer,  AT&T  Bell  Laboratories 

Hugo  Vifian,  Harmonic  Lightwaves 

Owen  P  Williams,  Motorola,  Inc. 

Submitted  for  the  panel  by  its  Chair,  V  Thomas  Rhyne,  this  assessment  of  the  fiscal  year 
1995  activities  of  the  Electronics  and  Electrical  Engineering  Laboratory  is  based  on  site  visits  by 
individual  panel  members,  a  formal  meeting  of  the  panel  on  February  15-17,  1995,  in  Boulder, 
Colorado,  and  on  documents  provided  by  the  laboratory. 


LABORATORY  OVERVIEW 

Mission 

The  mission  of  the  Electronics  and  Electrical  Engineering  Laboratory  (EEEL)  as  stated  in 
its  program  plan  is 

to  promote  economic  growth,  and  especially  international  competitiveness,  by 
providing  measurement  capability  of  high  economic  impact  focused  primarily  on 
the  critical  needs  of  the  US  electronics  and  electrical-equipment  industries.  In 
fulfilling  this  mission,  EEEL  strives  to  provide  leading-edge  capability  supportive 
of  each  of  the  major  steps  required  to  realize  competitive  products  in  the 
marketplace:  research  and  development,  manufacturing,  marketplace  exchange. 
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and  after-sales  support.  Good  measurement  support  is  essential  for  accelerating 
the  commercialization  of  technology,  a  primary  requirement  for  improved  U.S. 
competitiveness  (Electronics  and  Electrical  Engineering  Laboratory  J  995 
Program  Plan  [NISTIR  5563],  NIST,  Gaithersburg,  Md  ,  pp  1-2) 


Resources 

EEEL  funding  for  fiscal  year  1994  totaled  $45.2  million;  $29  4  million  of  this  total  was 
directly  appropriated  or  STRS  (Scientific  and  Technical  Research  and  Services)  ftinding;  $3.0 
million  came  fi^om  the  Advanced  Technology  Program  (ATP);  $10.2  million  came  fi-om  external 
sources,  mostly  other  government  agencies  (other  agency  or  OA  ftinding);  and  $2  7  million  came 
fi-om  fees  for  calibrations  and  other  services    Fiscal  year  1995  ftinding  was  estimated  at  the  time 
of  this  assessment  to  total  $46.0  million,  with  $30.7  million  fi^om  STRS,  $2.1  million  fi-om  the 
ATP,  $11.1  million  fi-om  OA,  and  $2. 1  million  fi-om  fees  for  calibrations  and  other  services.  Total 
staffing  of  322  includes  283  full-time  permanent  employees,  of  whom  215  are  technical 
professionals,  1 16  of  whom  hold  a  PhD  degree. 


The  Priority-Setting  Process 

EEEL  has  used  formal  priority-setting  methodologies  as  a  means  for  selecting  new 
projects  for  almost  two  decades    The  current  methodology  has  been  in  place  since  1991.  It 
assesses  current  and  proposed  projects  against  four  criteria:  fit  to  mission;  potential  impact  if 
successful,  probability  of  success,  and  impact  on  the  institutional  health  of  NIST,  EEEL,  and  the 
EEEL  staff.  The  objective  of  these  multidimensional  assessments  is  to  focus  available  EEEL 
resources  on  the  most  appropriate  set  of  technical  projects.  In  addition,  the  EEEL  priority-setting 
process  allows  for  consideration  of  a  limited  number  of  project  proposals  that  do  not  meet  these 
criteria  precisely  but  still  offer  the  potential  for  outstanding  technical  or  scientific  value. 

EEEL's  written  description  of  the  current  priority-setting  methodology  divides  each  of  the 
four  criteria  listed  above  into  more  specific  subcriteria.  The  "Fit  to  Mission"  criteria,  for  example, 
includes  assessment  of  the  project's  goals,  customers  served,  deliverables,  steps  served  within  an 
associated  manufacturing  process,  and  uniqueness    The  "Impact  if  Successfiil"  assessment 
includes  the  criteria  of  market  size,  market  immediacy,  timeliness,  criticality,  sensitivity,  visibility, 
and  commitment. 

In  current  practice,  proposals  for  new  projects  are  required  to  include  detailed  descriptions 
of  the  project  versus  the  criteria  and  subcriteria  defined  in  the  EEEL  priority-setting  process 
Reviews  of  new  projects  are  normally  conducted  as  part  of  EEEL's  annual  technical  planning  and 
budgeting  activities,  although  opportunities  are  provided  for  initiating  high-priority  projects  at 
other  times  if  the  assessment  of  such  projects  indicates  opportunities  for  significant  success  and 
impact. 


96 


Assessment  or  Priorities  and  the  Priority-Setting  Process 

The  selection  of  projects  within  EEEL  generally  appears  to  be  well  matched  to  current 
industrial  needs  in  the  areas  served  by  the  laboratory.  This  assessment  can  largely  be  attributed  to 
the  effectiveness  of  HEEL' s  formal  priority-setting  process    Given  limitations  in  available 
resources  and  the  breadth  and  depth  of  need  for  advancement  in  the  technical  and  scientific  areas 
associated  with  the  mission  of  the  laboratory,  however,  continuous  refinement  of  this  process  is 
essential  to  keeping  EEEL  in  touch  with  its  customers    With  this  goal  in  mind,  the  panel  believes 
there  is  need  for  more  involvement  of  and  cooperation  among  the  managers  of  all  EEEL  divisions 
in  setting  divisional  and  laboratory  priorities 

As  an  aid  to  evaluating  year-to-year  accomplishments,  EEEL  should  report  each  year's 
accomplishments  in  relation  to  objectives  and  milestones  for  that  year    Year-end  reporting  should 
identify  goals  that  were  accomplished  as  planned,  goals  that  were  not  reached  (and  explanations), 
and  additional  accomplishments  over  and  above  the  plan 

More  specific  comments  regarding  the  effectiveness  of  project  selection  within  the 
divisions  and  offices  are  provided  in  the  divisional  assessments  below.  Comments  regarding  the 
possible  need  for  new  projects  or  for  the  cessation  of  current  ones  can  be  interpreted  as 
indications  of  the  need  for  improvement  in  priority  setting  within  that  division    As  the  remainder 
of  this  report  indicates,  however,  such  instances  are  few,  and  the  consensus  assessment  of  the 
laboratory's  priority-setting  process  was  quite  favorable. 


Assessment  of  Technical  Programs 

The  panel  found  EEEL's  technical  programs  to  be  generally  effective  and  competent.  The 
laboratory's  greatest  challenge  is  to  establish  a  strategy  for  acquiring  new  skills  to  respond  to 
changing  needs    Constant  attention  to  better  links  between  EEEL's  Boulder,  Colorado,  and 
Gaithersburg,  Maryland,  groups  is  also  needed    More  effective  collaborations  might  be  gained  by 
increasing  awareness  of  EEEL's  competencies  throughout  all  of  NIST. 

The  panel  was  charged  with  reviewing  EEEL's  work  with  data  programs    EEEL  has  not 
been  active  in  NIST's  Standard  Reference  Data  (SRD)  Program,  although  the  Office  of 
Microelectronics  Programs  (OMP)  is  sponsoring  work  in  data  for  plasma  etching  and  chemical 
vapor  deposition  modeling  which  may  be  distributed  through  the  SRD  Program  in  the  future 
EEEL  has  not  been  active  in  data  programs  primarily  because  the  industrial  community  served  by 
EEEL  has  not  established  such  activities  as  a  high  priority    EEEL  has  independently  responded  to 
a  limited  number  of  highly  specific  requests  for  data,  acting  directly  with  interested  commercial 
and  governmental  users    Participation  in  the  SRD  Program  could  enhance  the  availability  of 
EEEL  data  collections  and  generate  additional  revenue  to  support  or  expand  those  activities    Any 
participation  should  be  based  on  demonstrated  need  of  EEEL  customers  for  such  resources 
Specific  examples  of  EEEL's  independent  data  collection  and  distribution  activities  are  given  in 
the  divisional  assessments  below. 
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Assessment  of  Impact  on  Industry 

The  panel  identified  numerous  examples  in  wiiich  the  work  carried  out  in  the  laboratory 
during  fiscal  year  1994  resulted  in  measurable  impact  on  one  or  more  sectors  of  U.S.  industry.  As 
a  general  comment,  however,  the  industrial  impact  of  EEEL's  technical  and  scientific  work  was 
most  concentrated  in  sectors  in  which  industries  have  identified  their  needs  for  EEEL's  services. 
An  example  of  this  is  the  semiconductor  industry,  which  has  explicitly  stated  its  metrology  needs 
in  The  National  Technology  Roadmapfor  Semiconductors  (Semiconductor  Industry  Association, 
San  Jose,  Calif,  1994,  commonly  known  as  the  SIA  Roadmap).  Recent  EEEL  efforts  to  assist 
other  key  industrial  sectors  in  identifying  their  present  and  fijture  needs  are  quite  appropriate  and 
likely  to  improve  the  impact  of  EEEL  work  in  those  sectors.  Specific  comments  on  programs  are 
provided  in  the  divisional  assessments  below. 

^^ST's  new  Industry  Fellows  Program  provides  opportunities  for  NIST  staff  members  to 
gain  research  experience  in  industrial  settings,  thus  enhancing  the  technical  and  scientific 
qualifications  of  key  NIST  staff  members  while  improving  staff  appreciation  of  industrial  R&D 
operations    EEEL  has  played  a  key  role  in  the  development  of  this  program  and  was  one  of  the 
first  NIST  laboratories  to  participate  in  it    Three  of  four  1995  NIST  industry  fellows  are  EEEL 
staff  members    The  panel  approves  highly  of  this  level  of  participation    The  implementation  of 
the  NIST  Industry  Fellows  Program  is  an  outstanding  success  based  on  feedback  from  both 
industry  and  NIST  participants    This  program  guarantees  the  ultimate  partnership  between  NIST 
and  its  customers,  and  it  should  be  expanded  to  smaller  companies  that  are  state-of-the-art 
performers  in  their  field    Given  the  need  for  closer  interactions  between  EEEL  staff  members  and 
the  industries  that  they  serve,  continued  participation  in  this  program  appears  to  be  an  excellent 
way  for  EEEL  to  improve  the  quality  of  its  technical  and  scientific  activities. 


Recommendations 

The  following  are  the  panel's  recommendations  for  EEEL  as  a  whole 

Working  closely  with  customers  in  both  industry  and  government,  EEEL  management  should 
review  data  collection  and  distribution  activities  to  determine  the  usefulness  of  participation  in 
the  NIST  Standard  Reference  Data  Program. 

EEEL  should  continue  to  aggressively  pursue  participation  in  the  NIST  Industry  Fellows 
Program,  including  work  with  small,  cutting-edge  companies,  and  should  consider  reciprocal 
arrangements  that  can  expand  opportunities  for  engineers  and  scientists  from  industry  to  work 
at  EEEL 

EEEL  should  expand  its  activities  to  assist  additional  industrial  groups  in  the  electronics  and 
electrical  engineering  sectors  in  defining  their  needs  for  metrology  and  related  services, 
paralleling  the  work  of  the  Semiconductor  Industry  Association  in  the  SIA  Roadmap. 

EEEL  should  seek  ways  to  make  its  Gaithersburg  and  Boulder  groups  work  more  closely 
together. 
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EEEL  should  seek  to  make  the  competencies  of  the  laboratory  better  known  and  understood 
across  all  of  NIST. 


Electronics  and  Electrical  Engineering  Laboratory 
Responses  to  Fiscal  Year  1994  Recommendations 

Below  are  the  panel's  1994  recommendations  for  EEEL  as  a  whole,  as  quoted  from  the 
fiscal  year  1994  assessment  report,  followed  by  the  laboratory's  response.  The  panel  found  these 
responses  generally  satisfactory 

1994  Panel  Recommendation:  "EEEL  technical  management  should  conduct  periodic  reviews  of 
fundamental  research  activities  to  verify  that  projects  have  a  strong  likelihood  of  benefiting  the 
laboratory's  mission  and  to  identify  new  areas  in  which  fundamental  research  is  critically  needed 
to  provide  sufficient  measurement  capabilities  for  advancing  calibration  and  repeatability 
requirements.  ..."  (p.  20). 

Laboratory  Response:  NlST's  role  as  the  nation's  premier  measurements  laboratory  requires 
EEEL  to  remain  at  the  forefront  of  electrical  measurements,  especially  the  measurement  of  basic 
electrical  quantities.  EEEL  believes  that  maintaining  a  metrology  leadership  position  is  the  best 
strategy  for  positioning  to  meet  new  calibration  needs  and  paradigms,  given  that  fundamental 
directed  research  projects  are  long-term  affairs  and  that  predictions  of  the  future  become  more 
uncertain  as  the  prediction  time  is  extended 

EEEL  does  conduct  internal  quasi-periodic  reviews  of  all  projects  at  about  3-year 
intervals;  these  include  5-year  assessments  of  project  plans    Each  division  provides  5-year  plan 
charts  of  its  projects  as  part  of  its  annual  Program  Plan.   Also,  laboratory  management  maintains 
a  general  oversight  of  project  activity  through  internal  reporting  mechanisms  and  as  a  result  of 
preparing  the  Program  Plan.  Individual  projects  are  reviewed  on  an  ad  hoc  basis  in  order  to 
resolve  specific  issues;  EEEL  will  consider  conducting  a  specific  strategic  review  of  fundamental 
directed  research  projects  in  calendar  year  1995 

J994  Panel  Recommendation:  "Procedures  for  assigning  technical  work  to  divisions  should  be 
reviewed  to  assure  that  work  is  assigned  most  appropriately.  Efforts  to  align  the  divisions' 
strengths  more  closely  with  the  global  EEEL  mission  should  continue"  (p  20). 
Laboratory  Response:  EEEL  has  developed  and  promulgated  among  its  staff  guidelines  for 
project  selection.  There  is  a  shift  away  from  projects  that  benefit  primarily  other  agencies 
However,  many  OA  projects  undertaken  in  EEEL  have  been  and  are  in  pursuit  of  goals  aligned 
with  the  EEEL  mission,  one  of  NlST's  responsibilities  is  to  assist  other  agencies  with  respect  to 
metrology,  and  OA  projects  are  a  means  to  keep  in  contact  with  advanced  developments. 
Implementation  of  the  panel's  suggestion  to  extend  strategic  planning  more  explicitly  to  the 
division  level  will  contribute  to  a  more  integrated  laboratory  structure. 

J99-4  Panel  Recommendation:  "EEEL  should  identify  less  organized  sectors  of  the  US 
electronics  and  electrical  engineering  industry,  which  seem  of  critical  importance  to  U.S. 
competitiveness,  and  provide  support  of  leadership  in  identifying  their  metrology  needs"  (p.  20). 
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Laboratory  Response:  Measurements  for  Competitiveness  in  Electronics  (NISTIR  4583,  U.S. 
Department  of  Commerce,  1993)  is  intended  to  provide  a  vehicle  through  which  EEEL  can 
identify  principal  measurement  needs.  A  number  of  significant  technical  areas  remain  to  be 
addressed.  Some  of  these  potential  constituencies  will  be  disaggregated  in  nature  and  lack  a 
cohesive  voice,  akin  to  the  situation  experienced  with  much  of  the  industry  providing 
measurement  instrumentation.  Widespread  participation  by  EEEL  staff  in  conferences, 
professional  societies,  standards  organizations,  and  trade  associations  helps  the  laboratory  to  be 
aware  of  needs. 

1994  Panel  Recommendation:  "As  NIST's  resources  increase,  EEEL  should  be  proactive  in 
obtaining  resources  for  work  in  basic  measurements  and  R&D  of  critical  importance  to  the  U.S. 
electronics  and  electrical  engineering  industries"  (p.  20). 

Laboratory  Response:  Obtaining  resources  for  such  work  is  part  of  budgetary  initiatives  under 
development  for  fiscal  year  1997  and  was  considered  as  part  of  unsuccessfijl  initiatives  for  fiscal 
year  1996.  In  addition,  EEEL  has  sought  and  been  awarded  competence  funds,  three  such 
projects  are  active  in  fiscal  year  1995. 

J 994  Panel  Recommendation:  "EEEL  staff  should  review  current  policies  regarding  support  of 
non-U.  S. -owned  industries,  since  the  technical  needs  of  foreign-owned  companies  may  be  of 
critical  importance  to  industrial  developments  in  the  United  States  and  to  agreement  between  U.S. 
standards  and  measurements  and  those  of  international  trading  partners  and  competitors"  (p.  20). 
Laboratory  Response:  This  matter  is  beyond  EEEL  control    Formal  restrictions  deriving  from 
statute  state  that  a  foreign-owned  company  must  certify  that  it  is  "incorporated  under  the  laws  of 
one  of  the  states  or  territories  of  the  United  States,  and  that  it  has  a  manufacturing  presence  in  the 
United  States,  and  that  the  foreign  government  [involved]  permits  United  States  agencies, 
organizations,  or  other  persons  to  enter  into  cooperative  agreements  and  licensing  agreements." 
EEEL  has  entered  into  a  Cooperative  Research  and  Development  Agreement  (CRADA)  with  at 
least  one  foreign-owned  company  that  satisfied  the  restriction. 

1994  Panel  Recommetidation:  "Opportunities  exist  for  more  complete  utilization  of  the  Baldrige 
Award  criteria  in  laboratory  planning  and  execution  The  panel  found  no  consistent  use  of  such  a 
system  within  NIST,  although  some  efforts  have  begun;  EEEL  has  no  strategic  plan  to  put  such  a 
system  in  place"  (p.  20). 

Laboratory  Response:  Complex  issues  are  involved  in  applying  the  criteria  to  a  research  and 
development  organization    In  EEEL,  the  Electricity  Division  serves  as  a  pilot  for  the  laboratory 
and  formed  a  team  drawn  from  all  levels  to  investigate  quality  issues.  This  past  year,  following 
visits  to  a  number  of  firms  to  see  total  quality  management  (TQM)  in  action  and  to  discuss  the 
application  of  TQM  principles,  the  division  hired  a  consulting  organization  that  has  conducted 
interviews  with  division  staff   The  division  is  now  evaluating  feedback  from  the  interviews  to 
decide  the  next  steps. 

J994  Panel  Recommendation:  "EEEL  should  leverage  resources  through  targeted  participation 
in  ATP  proposals.  This  effort  can  also  help  organize  industrial  participation  in  key  areas  of 
technology  development"  (p  20) 
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Laboratory  Response:  Guidance  recently  issued  by  the  NIST  director  states  that  "IsflST 
personnel  should  not  be  involved  in  ATP  proposal  preparation  and  should  not  critique  applicants' 
proposals."  The  same  guidance  states  that  "technical  collaboration  with  awardees  is  encouraged," 
under  the  provisions  of  a  CRADA.  EEEL  has  established  CRADAs  with  several  ATP  awardees 
and  will  continue  to  pursue  participation  in  CRADAs  with  awardees  when  there  is  a  mutual 
benefit. 

1994  Panel  Recommendation:  "EEEL  management  and  NIST  human  resources  personnel  must 
maintain  the  strength  of  the  technical  staff  as  the  role  of  EEEL  shifts  away  fi-om  some  long-term 
OA  customers  and  their  specialized  technical  requirements    Appropriate  retraining  opportunities 
should  be  provided  for  displaced  staff,  and  staff  with  new  capabilities  must  be  recruited"  (p.  20) 
Laboratory  Response:  In  fiscal  year  1994,  almost  one-third  of  the  staff  participated  in  some  216 
courses;  many  of  these  individuals  are  professionals,  and  of  these,  a  number  were  extending  their 
areas  of  competence  to  meet  new  metrology  needs  from  industry.  Also  in  fiscal  year  1994,  EEEL 
hired  27  persons,  including  a  number  of  well-qualified  professionals  to  strengthen  staff  skills  in 
new  areas. 


DIVISIONAL  ASSESSMENTS 

Electricity  Division 

Mission 

The  Electricity  Division  provides  the  fijndamental  basis  for  all  electrical  measurements  in 
the  United  States    The  division  supports  US  industry  by  providing  electrical  measurements  and 
standards  infrastructure  required  to  develop  new  products,  ensure  quality,  and  compete 
internationally. 

The  division  realizes  the  electrical  units  in  terms  of  the  International  System  of  Units  (SI) 
and  determines  fijndamental  constants  related  to  electrical  units    The  division  is  responsible  for 
providing  calibration  services  and  developing  and  improving  the  measurement  methods  and 
services  needed  to  support  electrical  materials,  components,  instruments,  and  systems  used  for 
conducted  electrical  power,  industrial  electronics,  and  electronic-related  products  and  services. 
The  division  is  responsible  for  facilitating  the  design,  manufacture,  documentation,  procurement, 
and  application  of  electronics  systems  by  supporting  development  of  product  data  exchange 
standards  and  specifications    The  division  is  also  responsible  for  the  development  of  measurement 
technology  to  support  electronic  imaging  and  video  products. 

Resources 

The  division's  fiscal  year  1994  funds  totaled  $10.3  million,  comprising  $5.7  from  STRS, 
$0.8  million  fi-om  ATP,  $1 .2  million  from  fees  for  calibrations,  and  $2.6  million  fi-om  OA  Fiscal 
year  1995  funding  was  estimated  at  the  time  of  the  assessment  to  total  $10.7  million:  $6.5  million 
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from  STRS,  $0.7  million  from  ATP,  $1.2  million  from  calibrations,  and  $2.3  million  from  OA.  Of 
a  total  paid  staff  of  80,  54  are  technical  professionals  and  24  hold  a  PhD  degree. 


Assessment  of  Priorities  and  the  Priority-Setting  Process 

In  most  areas,  the  prioritization  of  the  division's  projects  is  in  concert  with  industry 
requirements.  Participation  in  industry  associations  and  industry  visits  and  involvement  in 
international  activities  have  provided  the  division  with  a  solid  sense  of  industry  needs  for 
standards  and  calibration    However,  the  program  in  electric  power  is  experiencing  a  general 
decline  in  funding,  and  the  research  planned  in  this  area  seems  disjointed  and  suffers  from  a  lack 
of  industry  visibility  and  direction,  even  though  this  is  a  time  of  great  potential  opportunity. 

The  prioritization  process  seems  to  deal  best  with  evolutionary  programs    Program  plans 
in  new  areas,  such  as  video  technology  and  automated  electronics  manufacturing,  reflect  a 
significant  disparity  between  stated  goals  and  the  resources  actually  provided  to  achieve  these 
results. 


Assessment  of  Technical  Programs 

The  panel  found  the  technical  quality  of  the  division's  programs  to  be  consistently  good. 
In  some  cases  (for  example,  the  Video  Technology  project  and  the  Automated  Electronics 
Manufacturing  project),  resources  were  quite  limited  compared  with  the  technical  objectives. 
Careful  prioritization  is  required  in  these  areas,  and  perhaps  a  reduction  in  stated  objectives 
should  be  considered    Resources  could  become  an  issue  in  the  Measurements  to  Support  Electric 
Utilities  project  as  well,  if  tighter  linkages  with  industry  result  in  an  expansion  of  program  goals. 
Some  turnover  is  being  experienced  in  this  division,  and  aging  of  the  staff  and  continued 
dependence  on  retired  staff  are  concerns,  especially  in  the  area  of  fundamental  standards. 
Maintenance  of  fundamental  standards  is  an  exacting  and  unusual  specialty;  EEEL  needs  to  attract 
qualified  young  scientists  and  ensure  that  accumulated  knowledge  is  transferted  to  them  as  part  of 
an  orderly  replacement  and  renewal  process    The  division  has  continued  to  demonstrate  the 
ability  to  provide  the  fundamental  metrology  capabilities  that  are  its  mission  and  to  develop  new 
technology  and  enhanced  capability  in  these  areas    The  following  comments  on  key  projects 
illustrate  this  balance 

The  Generation  and  Measurement  of  Precise  Signals  project  and  the  Improved  Impedance 
Calibration  Service  project  are  closely  coupled.  Progress  over  several  years  has  replaced 
antiquated  measurement  systems  with  modem  technology  that  provides  enhanced  capability.  For 
example,  a  newly  developed  digital  impedance  bridge  has  allowed  the  first  significant 
improvement  in  inductance  measurements  in  40  years    This  new  technique  for  measurement  of 
inductance  will  also  lead  to  reduced  uncertainties  in  the  generation  and  measurement  of  precise 
signals    Based  on  this  advance,  as  well  as  on  improvements  in  the  binary  inductive  voltage 
divider,  digital  synthesis,  and  other  measurement  technologies,  decreases  in  uncertainty  by  factors 
of  1 0  or  more  are  targeted  for  calibration  services  for  capacitance,  inductance,  and  loss. 

The  AC/DC  Difference  Standards  and  Measurement  Techniques  project  may  soon  provide 
industry  with  thermal  transfer  standards  that  are  nearly  as  precise  as  the  national  standard.  These 


12 


102 


thermal  transfer  standards  are  necessary  in  measurement  of  electric  power,  energy,  alternating 
current  (ac)  voltage,  and  ac  current.  The  existing  national  standard  is  inadequate  to  calibrate  the 
transfer  standard  over  its  full  range.  In  conjunction  with  the  Semiconductor  Electronics  Division 
and  the  Electromagnetic  Technology  Division,  cryogenic  phenomena  are  being  used  to  develop  a 
new  thermal  converter  with  sensitivity  approaching  the  quantum  limit,  extending  the  range  and 
uncertainty  of  the  current  national  standard    The  division  shared  an  R&D  100  Award  with  the 
Semiconductor  Electronics  Division  and  Ballantine  Laboratories  (Cedar  Knolls,  N.J.)  for  work  in 
this  area,  and  the  importance  of  this  project  to  industry  is  reflected  by  the  large  number  of 
CRADAs  relating  to  this  work.  However,  lack  of  staff  has  caused  slower  than  desirable  progress 
on  this  project 

The  Resistance  Standards  and  Measurement  Methods  project  continues  with  a  high  rate  of 
success    Two  technological  improvements — a  guarded  scanner  under  development  in  industry 
with  NIST  support,  and  implementation  of  an  active-arm  bridge  (following  the  approach  of  the 
U.K.  National  Physical  Laboratory) — may  lead  to  automation  of  high-resistance  measurements 
and  may  extend  these  measurements  to  wider  ranges.  This  capability  is  needed  by  industry,  which 
requires  accurate  resistance  measurements  for  quality  and  process  control  purposes.  It  will  be  of 
particular  importance  to  the  semiconductor  industry  to  support  the  reliable  and  inexpensive 
testing  of  nonfluorocarbon  solvents  used  in  processing. 

The  Waveform  Acquisition  Devices  and  Standards  project  also  continues  to  improve 
capability  in  line  with  evolving  industry  requirements,  addressing  characterization  of  the 
performance  of  the  most  demanding  waveform  sampling  systems  across  a  broad  range  of  speeds 
and  resolutions    Results  from  this  program  are  delivered  in  the  form  of  measurement  services  and 
in  the  development  and  transfer  of  useful  measurement  standards  and  measurement  techniques. 
An  ongoing  challenge  is  the  creation  of  new  capabilities  for  generating  and  characterizing 
picosecond  pulses.  The  YAG  laser  technique  demonstrated  during  fiscal  year  1994  has  presented 
technical  issues  that  suggest  this  may  not  be  the  optimum  technology  to  pursue,  An  important 
objective  for  the  coming  year  should  be  completion  of  a  technological  assessment  of  this  and 
alternative  approaches  and  selection  of  a  technology  for  implementation    This  is  especially 
important,  since  NIST  capability  does  not  now  provide  a  comfortable  safety  margin  when 
compared  with  the  emerging  requirements  within  industry. 

The  Measurements  for  Complex  Electronic  Systems  project  has  led  to  techniques  for 
modeling  complex  measurement  acquisition  systems  that  offer  promise  for  simplifying  industrial 
manufacturing  test  processes  and  significantly  reducing  test  costs.  Two  companies  actively 
benefit  from  this  technology,  and  others  are  considering  the  approach.  Ongoing  efforts  to  reduce 
this  technology  to  a  set  of  software  tools  attuned  to  the  needs  of  industrial  users  are  a  critical  step 
in  deriving  the  maximum  benefit  from  this  work. 

The  Quantum  Voltage  and  Current  project  and  the  Quantum  Resistance  and  Capacitance 
project  provide  the  required  legal  basis  for  virtually  all  electrical  measurements  in  the  United 
States    Major  developments  in  the  recent  past  such  as  Josephson  array  voltage  standards  and 
resistance  standards  based  on  the  quantum  Hall  effect  have  made  positive  contributions  to  NIST's 
worldwide  reputation    Current  work  on  the  ac  quantum  Hall  effect  could  lead  to  a  primary 
impedance  standard  that  is  more  readily  usable  for  calibrations  than  the  current  standard. 
However,  the  priority  of  work  on  both  ac  and  direct  current  (dc)  quantum  Hall  effects  relative  to 
other  potential  projects  should  be  reviewed  to  assure  maximum  impact  by  these  researchers. 
Single-electron  tunneling  (SET)  research  ongoing  in  both  Boulder  and  Gaithersburg  provides  a 
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valuable  link  between  lithographic  techniques  and  primary  standard  development.  This  research 
could  lead  to  a  current  source  directly  derived  from  a  frequency  standard,  increasing  the  accuracy 
of  such  standards  by  allowing  intercomparison  with  current  sources  based  on  the  Josephson  effect 
and  the  quantum  Hall  effect.  Although  earlier  duplication  of  efforts  at  the  two  sites  has  been 
resolved,  management  must  give  continual  attention  to  keeping  each  location  focused  on  its 
primary  areas  of  competence.  Efforts  to  develop  an  "electronic  kilogram,"  that  is,  a  standard  for 
the  kilogram  based  on  electronic  properties  which  would  replace  the  current  artifactual  standard, 
have  been  recognized  with  NIST  competence  funding.  EEEL  should  regard  this  work,  which 
would  advance  the  kilogram  standard  to  an  accuracy  of  about  1  part  in  10  or  possibly  10  ,  as  a 
primary  challenge  and  complement  the  competence  funding  with  requisite  instrumentation  and 
personnel  support. 

The  Measurements  to  Support  Electric  Utilities  project  has  adequate  staff  and  facilities  to 
meet  its  current  objectives  but  should  consider  addressing  a  wider  range  of  utility  industry  needs. 
Current  work  in  the  areas  of  watt-hour  standards,  calibration  of  optical  current  transducers,  and 
reference  measurement  systems  for  high-voltage  testing  is  traditional,  appropriate,  and  of  high 
value  to  industry    The  division's  work  in  development  of  accurate  methods  for  measuring  low- 
frequency  electric  and  magnetic  fields  is  extremely  important  and  relevant  to  industry  needs;  here 
NIST  is  performing  a  vital  and  unique  service.  Ongoing  work  in  surge  protection  technology  is  of 
high  quality,  but  duplicates  work  being  performed  outside  of  NIST  and  does  not  seem  central  to 
the  mission  of  the  division    A  growing  issue  for  industry  is  revenue  metering  in  the  face  of 
increasing  waveform  distortion  brought  about  by  switching  power  supplies  and  other  nonlinear 
loads    In  general,  measurement  and  characterization  of  distorted  waveforms  is,  and  should  be,  a 
future  priority  for  the  division. 

Work  in  the  Dielectrics  Research  project  studying  the  generation  of  toxic  gases  in  SFg- 
insulated  high-voltage  circuit  breakers  has  been  important  in  laying  to  rest  concerns  in  this  area. 
As  a  related  follow-on,  work  on  use  of  partial  discharge  data  in  predicting  insulation  condition  is 
important    However,  a  much  better  connection  to  industry  and  domestic  equipment 
manufacturers  should  be  developed  if  this  work  is  to  be  continued    Also,  related  efforts  are 
ongoing  at  a  number  of  other  institutions,  making  the  uniqueness  of  NIST's  contribution  and 
capabilities  in  this  area  open  to  question 

The  Video  Technology  project  has  a  stated  objective  of  two  distinct  but  related  goals: 
developing  metrology  for  flat  panel  displays  and  developing  measurements  for  assessing  video 
quality  and  interoperability.  Unfortunately,  the  resources  of  this  team  do  not  appear  to  be 
sufficient  to  meaningfiilly  address  both  of  these  areas    The  practical  resuh  of  this  has  been  a 
convergence  within  the  group  on  work  primarily  addressing  flat  panel  displays  and  display-related 
perceptual  eflfects.  This  work  is  appropriate  and  should  continue.  However,  it  is  questionable 
whether  the  group  can  address  video  measurements  effectively  and  whether  a  window  of 
opportunity  still  exists  for  the  division  to  contribute  meaningfully  in  this  area. 

As  noted  in  the  fiscal  year  1 994  assessment,  the  Automated  Electronics  Manufacturing 
project  appears  to  be  critically  understaffed,  and  competition  for  resources  to  support  work  on 
product  data  standards  is  being  won  by  other  NIST  laboratories.  In  the  absence  of  a  significant 
infljsion  of  resources,  this  project  should  be  discontinued,  and  work  on  product  data  standards  for 
electrical  and  electronic  products  shifted  to  the  Manufacturing  Engineering  Laboratory,  which  has 
a  significant  program  in  exchange  of  product  data.  NIST's  role  in  this  area  is  appropriate,  and 
these  efforts  should  include  close  work  with  US.  electrical  and  electronic  industries,  but  a 
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subcritical  effort  within  this  division  does  not  appear  to  be  an  appropriate  use  of  EEEL's 
resources. 


Assessment  of  Impact  on  Industry 

The  technical  programs  of  the  division  are  having  a  positive  impact  on  industry,  as 
evidenced  by  the  use  of  calibration  services  and  industry  participation  in  training  and  workshops 
and  in  the  commercialization  of  research  through  CRADAs  and  other  forms  of  cooperation    An 
additional  indication  of  impact  on  industry  has  been  the  creation  of  a  Digital  Video  ATP  project  in 
response  to  industry  needs  identified  in  a  NIST  workshop    In  the  fiiture,  the  division's  industrial 
customer  base  may  be  expanded  to  include  users  and  manufacturers  of  flat  panel  displays  as  work 
in  this  area  matures.  However,  some  areas  provide  opportunity  for  improving  impact  on  industry, 
as  discussed  below 

Quality  systems  are  used  in  industry  to  improve  measurement  assurance,  competitiveness, 
effectiveness  of  research  and  development  activities,  and  the  quality  of  goods  and  services    No 
such  consistent  system  exists  within  NIST    The  Electricity  Division,  although  part  of  a  pilot 
program  for  use  of  quality  systems  within  NIST,  lacks  a  documented  or  consistent  quality  system 
and  has  not  shown  significant  progress  in  this  area  since  the  previous  assessment. 

New  intrinsic  standards  are  being  developed,  and  the  use  by  industry  of  existing  intrinsic 
standards  is  expanding.  These  intrinsic  standards  are  complex  and,  when  not  properly  operated, 
give  rise  to  significantly  erroneous  results    A  measurement  assurance  system  needs  to  be 
developed  to  preclude  degradation  of  the  US  measurement  system    The  division  should  also 
develop  plans  to  further  address  industry's  needs  for  seminars,  workshops,  and  publications  in  the 
areas  of  intrinsic  standards,  measurement  assurance,  and  measurement  techniques 

A  calibration  laboratory  accreditation  program  is  now  offered  by  NIST.  To  be  of 
maximum  value,  the  NIST  accreditation  program  must  be  accepted  internationally.  Such 
acceptance  requires  identification  and  resolution  of  technical  issues  related  to  differences  in  other 
national  programs,  such  as  development  of  a  formal  quality  system  for  calibration  services, 
specific  methods  of  developing  traceability  through  scaling,  and  formal  coordination  of  basic 
standards 

Industry  users  find  turnaround  time  for  NIST  calibration  services  excessive  The  division 
should  review  operating  processes  to  identify  opportunities  to  reduce  this  cycle  time.  In  addition, 
thorough  planning  must  be  done  in  advance  of  planned  facility  moves  to  avoid  disruption  of  these 
vital  services  to  industry. 

The  division  should  plan  for  the  effects  of  increasing  International  Organization  for 
Standardization  9000  series  (ISO  9000)  registrations    Calibration  requirements  in  this  standards 
series  are  more  stringent  than  those  generally  practiced  by  US  industry    This  increase  will  affect 
the  division's  workload  and  income  from  fees  and  will  also  require  continual  improvement  in 
these  services 

The  electric  power  industry  is  entering  a  period  of  great  change.  As  the  industry 
deregulates  in  a  manner  analogous  to  that  of  the  telephone  industry,  metering  and  measurements 
at  various  points  in  the  system  will  become  much  more  numerous  and  important  than  at  present. 
The  division  now  has  an  opportunity  to  assist  in  creating  needed  metering  and  measurement 
methods  and  standards.  This  is  all  the  more  important  because  of  the  changing  status  of  the 
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Electric  Power  Research  Institute  (EPRI)    Division  projects  related  to  electric  power  are 
individually  valuable  and  of  high  quality    However,  because  of  lack  of  focus,  these  projects  seem 
to  have  minimal  impact  on  industry    The  division  is  in  a  position  to  make  important  contributions 
if  it  can  achieve  the  right  focus  and  coupling  to  industry.  The  division  should  use  its  ties  to 
industry  players  such  as  EPRI,  the  Edison  Electric  Institute,  the  American  Public  Power 
Association,  the  Federal  Energy  Regulatory  Commission,  utility  companies,  independent  power 
producers,  and  electrical  equipment  manufacturers  to  develop  a  roadmap  of  metering  and 
measurement  needs  for  the  modem  electric  power  industry.  The  division  should  then  develop 
projects  based  on  that  roadmap  to  establish  and  maintain  its  technology  flow  to  industry. 


Recommendations 

The  following  are  the  panel's  recommendations  for  the  Electricity  Division. 

•  The  Electricity  Division  needs  to  attract  qualified  young  scientists  and  to  ensure  that 
accumulated  knowledge,  particularly  in  fundamental  standards,  is  transferred  to  them  as  part 
of  an  orderly  replacement  and  renewal  process. 

•  The  division  should  devote  sufficient  technical  effort  to  development  of  picosecond  pulse 
standards  to  ensure  identification  of  a  technology  path  for  short  pulse  measurement. 

•  The  division  should  resolve  the  inconsistency  between  program  goals  and  resources  in  the 
Video  Technology  project 

•  A  consistent  quality  system  should  be  developed  within  the  division  to  assure  the  quality  and 
effectiveness  of  services  delivered  to  industry  and  of  research  and  development  activities. 

•  The  division  should  develop  plans  to  ensure  the  integrity  of  intrinsic  standards  and  their  use. 

•  The  division  should  address  the  potential  impact  of  increasing  ISO  9000  registrations  and 
calibration  laboratory  accreditations    This  should  include  addressing  the  needs  of  U.S. 
industry  for  international  recognition  of  calibrations  performed  in  the  United  States. 

•  The  division  should  review  its  calibration  services  to  reduce  cycle  time  and  avoid  disruption  of 
service  during  facility  relocation. 

•  The  division  should  use  its  ties  to  key  players  in  electric  power  to  determine  a  more  focused 
program  to  serve  that  industry 


Electricity  Division  Responses  to  Fiscal  Year  1994  Recommendations 

The  panel's  1994  recommendations  for  the  Electricity  Division,  quoted  from  the  fiscal 
year  1994  assessment,  and  the  division's  response  are  given  below    The  1995  panel  was 
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disappointed  in  the  division's  responses  to  these  recommendations,  finding  them  somewhat . 
defensive  rather  than  responsive. 

1994  Panel  Recommendation:  "The  Electricity  Division's  mission  statement  should  mention 
fundamental  goals  of  its  work  and,  since  it  does  not  currently  include  the  entire  scope  of  the 
division's  responsibilities,  should  be  written  with  a  wider  focus"  (p  25). 

Division  Response:  The  mission  statement  was  rewritten  with  broader  scope 

1995  Panel  Comment:  The  mission  was  appropriately  modified  as  recommended    However,  it  is 
questionable  whether  the  division  has  the  resources  to  fially  discharge  this  mission,  especially  as  it 
relates  to  video  and  to  product  data  exchange.  It  may  be  appropriate  to  reduce  the  scope  of  the 
mission  in  these  areas. 

1994  Panel  Recommeruhtion:  "  .    Changes  in  the  structure  of  the  electric  utility  industry  and  its 
research  arm,  the  Electric  Power  Research  Institute  (EPRI),  provide  a  window  of  opportunity 
enabling  the  division  to  serve  this  industry's  needs  by  (1)  broadening  activities  to  provide  services 
to  new  players  and  stakeholders  in  the  power  industry  ...  (2)  improving  technology  transfer  to 
the  industry  ...  (3)  widening  the  use  of  collaborative  arrangements  with  the  industry,  and  (4) 
developing  new  customers  for  metrology  services  such  as  independent  power  producers  and  large 
industrial  customers"  (p.  25). 

Division  Response:  The  division  is  addressing  these  problems  and  opportunities  by  attempting  to 
direct  the  electric  power-oriented  portion  of  its  research  program  to  areas  that  most  critically 
benefit  the  industry.  The  ongoing  reorganization  of  the  division  allows  a  more  focused  effort  to 
be  applied;  specifically,  the  newly  formed  Electrical  Systems  Group  will  concentrate  most  of  its 
resources  on  utility-related  problems    However,  limitations  in  fiinding,  and  the  lack  of  a  clear 
determination  of  the  metrological  needs  of  the  electric  power  industry,  will  continue  to  restrict 
support  to  this  industry    In  an  attempt  to  determine  industry  needs,  division  and  laboratory 
management  will  be  visiting  industry  representatives    Different  approaches  to  technology  transfer, 
such  as  technical  workshops  and  collaborative  agreements,  are  being  taken  to  supplement  the 
traditional  publication  of  results. 

1995  Panel  Comment:  The  division  has  not  adequately  addressed  this  issue.  The  division  should 
use  its  ties  to  key  players  in  electric  power  to  determine  a  more  focused  program  to  serve  this 
industry. 

1994  Panel  Recommendation:  "The  panel  recommends  that  non-U  S  -owned  equipment 
manufacturers  have  access  to  research  results,  since  such  research  would  be  used  to  develop,  aid, 
or  improve  technology  in  U.S.  plants"  (p.  26) 

Division  Response:  Placing  limitations  on  access  to  MIST  results  by  non-U. S  -owned  companies 
is  a  policy  issue  beyond  the  division's  control    However,  nearly  all  of  the  division's  output  is 
published  in  the  open  literature. 

1994  Panel  Recommendation:  "The  division  should  establish  programs  in  areas  in  which  it  can 
play  a  valuable  roie,  such  as    .    energy  efficiency  .  .  .  electromagnetic  fields  .  .  .  power  quality 
.  .  .  distribution  and  transmission  systems  .  .  .  environmental  impacts  of  the  utility  industry,  and 
.  .  .  industry  .  .  .  benchmarking  standards"  (p  26) 
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Division  Response:  The  division  has  begun  a  program  funded  by  the  Department  of  Energy 
(DOE)  to  review  test  procedures  and  specifications  for  energy-efficient  motors  and  transformers 
The  division's  experimental  program  in  metrology  of  electromagnetic  fields  continues  essentially 
unchanged  fi"om  previous  years.  The  power  quality  program  continues  to  be  effective  in 
educating  utilities  and  industries  in  the  complex  area  of  power  quality;  no  calibration  services  are 
being  planned  at  this  time    The  division  continues  to  develop  its  program  to  provide  metrological 
support  of  optical  sensors  for  electric  power  transmission  and  distribution  systems. 
Benchmarking  of  measurement  practices  used  in  the  power  industry  would  require  resources 
considerably  beyond  those  of  the  division  at  present    Experience  elsewhere  in  EEEL  has  shown 
that  the  selection  of  a  standardized  set  of  measurement  parameters  and  the  subsequent  evaluation, 
analysis,  and  selection  of  a  set  of  methods  for  determining  those  parameters  are  complex 
processes  requiring  staff-years  of  effort  even  when  the  field  is  narrowly  defined 

199-i  Panel  Recommendation:  "Techniques  being  developed  by  the  division  for  measurements  in 
complex  electronics  systems  are  ready  for  practical  application  and  ready  to  be  assessed  for 
obstacles  to  implementation  and  economic  and  technical  benefits    The  division  should  consider 
technology  transfer  to  a  target  group  of  early  users  and  should  conduct  a  technical  and  economic 
analysis  of  the  results.  .  .  ."  (p.  26). 

Division  Response:  Efforts  to  transfer  information  on  testing  strategies  and  principles  have 
continued  by  visits  by  the  technical  staff  to  various  instrumentation  and  semiconductor  device 
manufacturers,  the  hosting  of  the  third  "Testing  Strategies  for  Analog  and  Mixed-signal  Products 
Workshop,"  and  the  initiation  of  development  of  a  user-friendly  software  tool  for  implementing 
the  matrix  operations  needed  to  develop  testing  strategy    Analog  Devices  (Wilmington,  Mass.)  is 
adopting  the  NIST  testing  strategies  for  its  data  converter  products    That  company's  economic 
assessment  thus  far  shows  a  projected  annual  savings  of  $400,000  on  the  production  testing  of 
two  of  its  16-bit  converter  products  and  a  cost  reduction  of  several  million  dollars  per  year  if  this 
approach  can  be  broadly  applied  to  its  converter  product  line    In  addition,  software  based  on 
NIST  testing  strategies  is  marketed  by  a  major  manufacturer  of  semiconductor  device  test 
equipment 

1994  Panel  Recommendation:  "At  this  early  stage,  work  on  metrics  for  video  quality  can  benefit 
fi^om  defining  the  problems  that  this  project  will  address    Particular  attention  should  be  given  to 
quality  metrics  required  for  successful  creation  and  operation  of  video  for  the  Nil  [national 
information  infi-astructure],  and  how  the  proposed  metrics  will  be  employed  by  industry"  (p.  26). 
Division  Response:  A  needs  assessment  for  video  quality  metrics  associated  with  compression 
techniques  is  being  conducted  through  review  of  published  literature,  telephone  conversations 
with  industry  representatives,  and  site  visits  to  selected  laboratories    This  assessment  will  identify 
specific  observable  artifacts  deemed  critical  by  industry  which  can  be  quantified,  so  the  resultant 
metric  can  be  used  in  part  to  define  quality    The  division  organized  a  workshop  attended  by  187 
industry  and  government  representatives  to  discuss  the  role  of  digital  video  within  the  national 
information  infrastructure 

1995  Panel  Comment:  Defining  and  staffing  the  program  in  video  have  not  been  adequately 
addressed.  At  this  time,  it  still  appears  that  the  stated  goals  and  the  amount  of  staff  assigned  are 
incompatible. 
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1994  Panel  Recommendation:  "Resources  allow  the  division  an  expanded  role  in  measurement 
applications.  This  will  require  closer  linkage  to  industrial  applications  and  partners  in  order  to 
identify  requirements  and  manage  the  transfer  of  technology,  faster  time  frames  in  order  to  meet 
industry  requirements  and  match  the  rate  of  change  of  industrial  technology,  and  establishment  of 
new  technical  competencies  as  required  by  targeted  measurement  applications"  (p.  26) 
Division  Response:  In  fact,  the  division  finds  itself  resource  limited    In  particular,  anticipated 
funding  to  support  new  areas  such  as  the  Video  Technology  project  and  the  Automated 
Electronics  Manufacturing  project  has  not  been  allocated.  In  all  work,  the  division  strives  to 
respond  in  a  timely  fashion  and  to  develop  the  necessary  competencies  to  meet  the  needs  of 
customers. 

1994  Panel  Recommendation:  "EEEL  should  ensure  that  the  Automated  Electronics 
Manufacturing  Program  receives  the  1995  resources  needed  to  accomplish  its  strategic  plans" 
(p.  26). 

Division  Response:  Division  and  laboratory  management  are  addressing  flinding  for  the 
Automated  Electronics  Manufacturing  Program  (AEMP).  An  increase  in  base  funds  was  sought 
in  the  fiscal  year  1 995  and  1 996  initiative  development  process,  but  in  each  instance  AEMP  was 
not  selected  for  funding.  An  increase  for  AEMP  is  again  being  sought  in  the  fiscal  year  1997 
initiative  process.  The  AEMP  will  be  getting  only  a  small  fraction  of  the  money  originally 
budgeted  for  fiscal  year  1995.  This  has  significantly  impacted  the  ability  of  the  AEMP  to  meet 
identified  needs  of  the  electronics  industry 

1994  Panel  Recommendation:  "The  division  should  continue  its  thrust  in  high-speed 
instrumentation,  establishing  ultraprecise  and  ultrahigh-speed  pulse  measurement  capability. 
Optoelectronic/electro-optic  technology  is  the  best  choice  to  provide  absolute  performance 
beyond  commercial  systems.  The  division  should  pursue  further  development  in  this  area  with  the 
goal  of  characterizing  the  best  commercial  and  developmental  waveform  sources  and  acquisition 
systems"  (p.  26). 

Division  Response:  Present  efforts  are  aimed  at  reducing  the  uncertainties  in  measuring  fast  pulse 
sources  with  transition  durations  down  to  1 5  picoseconds  and  sampling  systems  with  bandwidths 
up  to  20  GHz.  The  research  exploits  optoelectronic  means  for  generating  fast  electrical  pulses 
that  can  be  used  to  characterize  the  impulse  response  for  the  20  GHz  bandwidth  sampling  systems 
currently  used  for  supporting  the  NIST  fast  pulse  calibration  services    The  division  intends  to 
extend  this  effort  to  find  ways  for  supporting  state-of-the-art  50  GHz  bandwidth  sampling 
systems,  using  optoelectronic  means,  electro-optic  methods,  nonlinear  planar  transmission  lines, 
or  a  combination  of  all  three  technologies    At  the  same  time,  the  division  is  developing  sampling 
comparator-based  waveform  acquisition  systems  that  are  highly  accurate  for  characterizing  the 
settling  performance  of  the  best  commercial  data  converters  and  precision  waveform  sources. 

1995  Panel  Comment:  The  division  has  made  a  commitment  to  create  a  picosecond  pulse 
measurement  capability  as  recommended.  The  technology  path  has  proven  more  difficult  than 
anticipated  a  year  ago,  with  the  result  that  absolute  progress  is  less  than  expected.  Focus  should 
be  maintained,  with  an  initial  goal  of  selection  of  an  appropriate  technology.  The  alternative 
technologies  now  being  considered  may  be  more  suitable  to  replication  and  transfer  to  users  in 
industry. 
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Semiconductor  Electronics  Division 

Mission 

The  Semiconductor  Electronics  Division  develops  and  evaluates  measurement  methods, 
data,  reference  artifacts,  models  and  theory,  and  associated  technology  to  remove  market  barriers 
associated  with  the  use  of  semiconductor  materials  and  with  the  design,  fabrication, 
characterization,  and  performance  assurance  of  semiconductor  devices  and  integrated  circuits, 
conducts  research  in  semiconductor  materials  and  processes,  devices,  and  integrated  circuits  to 
provide,  through  both  experimental  and  theoretical  work,  the  necessary  basis  for  understanding 
measurement-related  requirements  in  semiconductor  technology,  and  disseminates  and  fosters 
application  of  technical  resuhs  and  assists  in  the  development  of  standardized  test  methods  and 
Standard  Reference  Materials  (SRMs)  to  enhance  the  quality,  performance,  and  reliability  of 
semiconductor  materials,  devices,  and  integrated  circuits  to  aid  in  manufacturing  productivity  and 
in  the  development,  transfer,  and  exploitation  of  semiconductor  technology  for  public  benefit. 

Resources 

Total  fiscal  year  1994  funding  of  the  Semiconductor  Electronics  Division  was  $8.03 
million,  comprising  $5.97  million  fi-om  STRS,  $0.78  million  fi-om  ATP,  $0.09  million  fi-om  fees 
for  SRMs,  and  $1.19  million  fi-om  OA    Fiscal  year  1995  was  estimated  at  the  time  of  the 
assessment  to  be  $9.04  million  total;  $7.65  million  fi-om  STRS,  $0.44  million  fi-om  ATP,  $0.06 
million  from  SRM  fees,  and  $0.89  million  from  OA.  Total  staffing  is  51.2  fijll-time  equivalents;  of 
these  positions,  34.8  are  technical  professional.  Of  42  technical  professional  staff  members,  25 
hold  a  PhD  degree. 

Assessment  of  Priorities  and  the  Priority-Setting  Process 

The  panel  found  it  difficult  to  assess  the  effectiveness  of  the  prioritization  process  in  this 
division  because  no  major  new  projects  were  initiated  or  terminated  since  the  previous 
assessment,  although  some  new  tasks  were  added  to  existing  projects    As  best  identified, 
however,  divisional  priorities  seem  to  favor  fijture  technologies  and  core  competencies    It 
appears  that  the  prioritization  between  work  in  the  key  areas  of  compound  semiconductors  and 
silicon  "shrink"  technology  has  been  applied  in  a  fairiy  informal  fashion,  especially  considering 
current  expertise  within  the  division,  available  facilities,  interests  of  current  personnel,  and  known 
interests  of  potential  customers     Also,  the  division's  mission  statement  does  not  convey  the  fiill 
breadth  of  its  role.  In  particular,  the  trend  toward  integration  and  the  division's  programs 
addressing  that  trend  are  not  reflected,  neither  are  the  division's  unique  metrology  capabilities  for 
measuring  materials  and  material  parameters  for  customers. 
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Assessment  of  Technical  Programs 

The  Nanoelectronics  project  responds  to  needs  identified  in  the  SIA  Roadmap  for 
measurements  on  the  scale  of  10  to  100  run    The  project's  emphasis  on  shallow  junction  dopant 
profiling  with  less  than  10  nm  spatial  resolution  is  appropriate,  for  without  this  capability, 
engineers  must  resort  to  trial  and  error  based  solely  on  electrical  test  data  to  verify  fine  line  design 
and  the  integrity  of  new  processes  for  manufacturing  submicrometer  structures.  Work  towards 
these  goals  using  scanning  capacitance  microscopy  is  world  class,  well  equipped,  and  in  direct 
support  of  EEEL's  metrology  mission    Development  of  techniques  for  in  situ  diagnostics  during 
molecular  beam  epitaxial  growth  of  semiconductors  is  also  within  the  metrology  charter,  but  this 
motivation  does  not  seem  to  be  at  the  heart  of  ongoing  investigations  of  growth  kinetics  within 
this  project.  The  division's  fijndamental  studies  of  these  materials  seem  to  duplicate  activities  in 
industry  and  universities,  and  sensor  accessories  seem  peripheral  to  the  ongoing  work 
Justification  for  work  on  compound  semiconductors  is  also  unclear;  this  portion  of  the  project 
seems  to  rely  heavily  for  guidance  on  the  SIA  Roadmap,  which  does  not  deal  explicitly  with 
compound  semiconductors    In  the  absence  of  other  guidance,  selection  of  materials  systems  for 
study  appears  more  by  circumstance  (such  as  an  existing  competency,  customer  involvement,  or 
available  ftinding)  than  by  strategic  intent    Work  with  x-ray  techniques  (diffraction,  reflection, 
fluorescence)  for  in  situ  monitoring  does  fill  an  existing  void  for  both  silicon  and  compound 
semiconductors    However,  the  division  needs  to  ensure  vendor  involvement  during  development 
so  that  tools  of  the  most  practical  value  become  commercially  available,  the  extent  of  this  activity 
needs  to  be  tempered  by  the  relatively  small  market  for  such  equipment 

The  division's  description  of  the  needs  addressed  by  the  Semiconductor  Characterization 
project  covers  applications  extending  from  digital  circuits  for  computers  to  emitters  for  optical 
communication.  This  area  is  extremely  broad    Of  the  nine  programs  within  the  project,  four 
relate  to  both  silicon  and  compound  semiconductors,  two  to  the  development  and  certification  of 
SRMs,  two  to  fiature  silicon  devices,  and  one  to  a  survey  to  identify  industrial  requirements  for 
techniques  and  related  reference  materials    Considering  the  current  problems  in  silicon  device 
characterization  in  the  250  to  100  nm  range,  it  is  surprising  that  no  programs  address  these  needs. 
The  panel  questions  this  and  urges  reassessment  of  program  priorities    In  addition,  progress  and 
industrial  impact  might  be  increased  if  this  work  were  organized  according  to  industry  needs  being 
addressed,  for  example,  into  programs  on  silicon  semiconductor  characterization  for  logic  and 
memory  and  on  compound  semiconductor  characterization  for  optoelectronic  communications. 

The  goals  of  the  Electrical  and  Thermal  Characterization  project  have  been  well 
articulated  to  provide  support  for  the  development  of  computer-aided  design  tools,  specifically 
electrical,  thermal,  and  mechanical  models  for  semiconductor  devices  and  packages,  as  identified 
by  the  SIA  Roadmap    The  technical  approach  to  be  taken — developing  the  theoretical 
foundations,  standards,  model  validation  procedures,  and  experimental  techniques  for 
measurement  of  chip  and  package  temperature  and  electrical  and  thermal  characteristics — is 
sound 

The  goals  of  the  Thin  Film  Characterization  project  are  twofold    establishment  of 
metrology  tools  for  evaluation  of  robust  gate  dielectrics  at  a  thickness  range  of  5  to  200  nm,  and 
development  of  low-cost  silicon-on-insulator  (SOI)  substrates    The  first  goal  is  aligned  with  the 
SIA  Roadmap,  and  the  second  is  motivated  by  the  need  to  support  an  Advanced  Research 
Projects  Agency  (ARPA)  low-power  electronics  initiative    The  technical  approach  is  to  develop 
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SRMs  and  transfer  this  technology  to  industry  with  traceability  to  NIST,  following  a  proven  track 
for  success 

Current  program  plans  for  thin  oxide  films  are  consistent  with  the  metrology  mission.  The 
commitment  to  developing  SRMs  in  the  range  of  1 0  to  200  nm  to  be  produced  by  industry  with 
NIST  support  is  a  timely  response  to  the  needs  of  the  semiconductor  industry. 

The  Test  Structure  Metrology  for  Advanced  Semiconductor  Manufacturing  project  seeks 
to  provide  domestic  semiconductor  manufacturers  with  improved  test  structures,  test  methods, 
and  diagnostic  procedures,  allowing  US  companies  to  manufacture  competitive,  cost-effective 
semiconductor  products.  The  project's  current  work  on  improved  methods  for  measuring 
linewidths  and  feature  placement,  for  developing  sensors  for  improved  process  control  in  the 
manufacture  of  interconnects  and  oxides,  for  improving  measurements  associated  with  micro- 
electro-mechanical  systems  (MEMS),  and  for  improving  tool  performance  through  better  gas 
sensor  arrays  are  appropriate  to  the  mission  of  the  division    Plans  to  improve  linewidth 
measurements  (a  collaboration  with  the  Manufacturing  Engineering  Laboratory),  to  evaluate 
contactless  measurements  of  capacitance  and  inductance  from  chemically  and  mechanically 
polished  interconnect  levels,  to  better  characterize  time-dependent  dielectric  breakdown  (TDDB) 
of  9  and  1 5  nm  oxides  under  electrical  stress  (a  collaboration  with  the  National  Semiconductor 
Corporation),  and  to  undertake  other  projects  including  collaboration  with  other  companies  and 
government  agencies  also  align  with  the  needs  identified  in  the  SIA  Roadmap. 


Assessment  oflmpact  on  Industry 

Accomplishments  and  impact  since  the  previous  assessment  in  the  Electrical  and  Thermal 
Characterization  project  were  primarily  in  the  area  of  power  devices  and  power  integrated 
circuits    These  results  are  of  importance  to  the  power  electronics  industry  both  fi-om  the 
component  supplier  side  and  the  user  side    This  work  also  provides  a  valuable  service  to  the 
sizable  industry  involved  with  appliance  controls,  heating/ventilating/air  conditioning,  factory 
automation  controls,  and  the  nascent  electric  vehicle  technology    Its  impact  can  be  measured  on 
the  basis  of  strong  industry  participation  in  NIST-sponsored  workshops  on  model  validation  and 
the  interactions  with  industry  via  the  CRADA  and  ATP  programs. 

Accomplishments  since  the  previous  assessment  in  the  Thin  Film  Characterization  project 
include  the  demonstration  of  interiaboratory  capability  for  thin  film  oxide  measurements  to  meet 
the  requirements  of  the  SIA  Roadmap  and  the  development  of  the  first  10  nm  oxide  thin  film 
SRM  in  response  to  industry  requests    These  milestones  are  important  and  demonstrate  the 
effective  industrial  impact  of  this  project    The  impact  of  this  project  is  also  evident  fi^om  the 
transitioning  of  SRM  technology  to  industry  to  meet  the  requirements  of  the  SIA  Roadmap. 

Since  the  previous  assessment,  the  Test  Structure  Metrology  for  Advanced 
Semiconductor  Manufacturing  project  demonstrated  the  dynamic  precision  of  electrical  overlay 
test  structures  to  be  1.5  nm,  improved  understanding  of  TDDB  in  thin  silicon  dioxide  films, 
refined  methods  of  electrically  measuring  the  thickness  of  metal  films,  and  fabricated  and  tested  a 
large  area  MEMS  thermal  display    These  and  other  accomplishments,  including  several  patents 
granted  within  this  projects,  offer  much  needed  help  to  U.S.  semiconductor  manufacturers  seeking 
better  control,  higher  accuracy,  and  higher  yields  within  their  fabrication  facilities. 
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Recommendations 

The  following  are  the  panel's  recommendations  for  the  Semiconductor  Electronics 
Division 

•  The  Semiconductor  Electronics  Division  should  revise  its  mission  statement  so  that  it  more 
accurately  reflects  the  division's  role  and  services 

•  The  division  should  consider  separating  its  work  on  semiconductor  characterization  according 
to  industry  needs  served 


Semiconductor  Electronics  Division  Responses  to  Fiscal  Year  1994  Recommendations 

The  panel's  1994  recommendations  for  the  Semiconductor  Electronics  Division  are  given 
below,  quoted  from  the  fiscal  year  1994  assessment  report,  with  the  division's  response.  The 
1995  panel  was,  in  general,  somewhat  disappointed  in  the  division's  response  to  these 
recommendations 

1994  Panel  Recommendation:  "The  panel  reiterates  its  fiscal  year  1993  recommendation  that  the 
Semiconductor  Electronics  Division  take  leadership  in  formulating  with  industry  a  national 
strategic  plan  for  compound  semiconductors"  (p  31). 

Division  Response:  The  division  has  actively  participated  in  the  development  of  the  second 
edition  of  the  National  Technology  Roadmap  for  Semiconductors  for  silicon  technology  and  has 
initiated  a  process  to  address  strategic  planiiing  for  compound  semiconductors    The  first  major 
activity  in  this  endeavor,  "Satellite  Workshop    Planning  for  Compound  Semiconductor 
Technology,"  was  held  in  conjunction  with  the  "International  Workshop  on  Semiconductor 
Characterization:  Present  Status  and  Future  Needs"  Participants,  in  addition  to  NIST,  included 
the  Optoelectronics  Industry  Development  Association,  ARPA,  the  National  Electronics 
Manufacturing  Initiative  (of  the  Electronics  Subcommittee,  Committee  on  Civilian  Industry  and 
Technology,  National  Science  and  Technology  Council),  and  other  industrial  technologists. 

1994  Panel  Recommendation:  "NTST,  with  the  support  of  the  SI  A  Roadmap  Coordination 
Group,  is  establishing  a  Semiconductor  Metrology  Advisory  Group  composed  of  government, 
industry,  and  university  interests  and  equipment  suppliers    NIST  should  use  this  advisory  group 
to  establish  priorities  for  the  metrology  needs  expressed  in  the  renewed  SI  A  Roadmap"  (pp  31- 
32) 

Division  Response:  This  advice  would  lead  to  complications  involving  the  Federal  Advisory 
Committee  Act.  The  division  concurs  that  it  is  essential  for  NIST  to  have  more  information  on 
the  industry's  metrology  needs  than  the  SIA  Roadmap  can  provide    Ways  of  accomplishing  this 
goal  have  been  developed  with  the  cooperation  of  the  SIA  Roadmap  Coordination  Group,  but 
details  of  implementation  are  still  being  worked  out 

1994  Panel  Recommendation:  "In  response  to  OA  fianding,  the  division  often  undertakes 
research  that  falls  outside  its  primary  metrology  mission    The  format  for  the  division's  program 
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plan  should  be  changed  to  relate  each  project  activity  clearly  to  its  funding  source,  thereby 

facilitating  a  better  assessment  of  project  objectives  and  accomplishments  in  relation  to  the 

appropriate  strategic  mission"  (p  32). 

Division  Response:  The  division  does  not  generally  take  on  OA  assignments  without  a  metrology 

component.  All  current  internally  and  OA-supported  activities  meet  the  EEEL  criteria  for  project 

selection.  Funding  sources  are  listed  in  the  division's  program  plan. 

J995  Panel  Comment:  Although  the  division  states  it  does  not  take  on  OA  assignments  without  a 

metrology  component,  the  panel  still  concludes  that  several  current  OA  projects  (such  as  projects 

on  micromachined  semiconductor  devices  and  separation  by  implanted  oxygen  material 

development)  fall  outside  the  primary  metrology  mission  and  strongly  urges  that  all  OA  projects 

be  measured  by  the  same  criteria  as  the  STRS  projects. 

J 994  Panel  Recommendation:  ".  .  .  [Projects]  that  were  supported  under  OA  funds  must  be 
evaluated  to  assure  that  their  objectives  can  be  clearly  related  to  the  strategic  metrology  mission 
of  the  division  before  STRS  ftjnds  are  applied  to  them    Examples  of  projects  that  need 
reevaluation  are  the  silicon-on-insulator  effort  and  model  development  for  commercial  software" 
(P  32). 

Division  Response:  The  SOI  effort  has  in  the  past  responded  mainly  to  OA  needs,  one  of  NlST's 
responsibilities,  and  is  still  largely  supported  by  funds  transferred  from  other  agencies,  even 
though  its  use  in  the  commercial  arena  for  future  circuits  seems  to  be  increasing.  The  SIA 
Roadmap  acknowledges  that  the  development  of  low-cost  SOI  substrates  would  advance 
integrated  circuit  performance  a  full  generation 

The  SIA  Roadmap  identifies  integration  of  component-level  electrical,  thermal,  and 
mechanical  models  for  semiuDnductor  devices  and  packages,  and  the  availability  of  calibrated  and 
easy-to-use  technology-computer-aided-design  tools  for  device,  process,  and  circuit  simulation  as 
areas  requiring  development  and  support  to  achieve  the  1 5-year  goals  of  the  Roadmap    The 
division  addresses  these  needs  by  developing  the  theoretical  foundations,  standards,  model 
validation  procedures,  and  associated  experimental  techniques  for  the  measurement  of  chip 
temperature,  device  electrical  and  thermal  characteristics,  and  package  electrical  and  thermal 
characteristics 

The  division  is  developing  with  industry  procedures  for  validating  device  models  for 
circuit  simulation    NIST  is  also  developing  procedures  for  characterizing  the  thermal  and 
electrical  performance  of  advanced  integrated  circuits  and  devices  and  the  associated  electronic 
packages  that  are  compatible  and  useful  for  computer-aided  design  of  boards  and  systems 
1995  Panel  Comment:  The  panel  feels  that  the  division's  response  misses  the  key  point  of  the 
recommendation 

1994  Panel  Recommendation:  "Because  of  rapid  changes  in  the  metrology  needs  of  the 
semiconductor  industry,  the  division  must  develop  an  approach  to  rapidly  acquiring  new  expertise 
as  such  expertise  becomes  necessary    The  panel  recommends  that  the  division  acquire  this 
expertise  from  industry,  university  laboratories,  and  other  government  laboratories  by  fianding 
sabbaticals  at  NIST  for  the  required  experts  when  the  need  arises  for  their 
competence.      ."  (p  32). 

Division  Response:  Expertise  is  acquired  through  CRADAs  and  collaborations  with  industry, 
universities,  other  government  laboratories,  national  laboratories,  and  consortia  and  through 
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faculty  hires  (including  sabbaticals)  and  students  and  guest  researchers  from  universities,  other 
government  laboratories,  and  the  semiconductor  industry    The  challenge  for  the  division  is  not  to 
necessarily  increase  the  number  of  collaborations  but  to  effectively  manage  them  to  maximize  the 
return  on  investment  of  time,  staff,  and  resources  by  both  parties. 

1995  Panel  Comment:  The  panel  reiterates  that  acquiring  new  expertise  and  increasing  ability  to 
respond  to  market  needs  is  the  greatest  challenge  facing  EEEL  and  the  division  in  the  immediate 
future.  A  well-articulated  strategy  on  how  EEEL  and  the  division  plan  to  increase  agility  is 
needed. 

1994  Panel  Recommendation:  "The  panel  endorses  the  strategy  to  allocate  a  small  fraction        of 
STRS  funds  to  basic  science  research    However,  these  activities  should  be  evaluated  yearly  for 
their  contribution  to  the  strategic  metrology  mission.  Basic  research  activities  that  do  not  show 
relevance  to  the  NIST  metrology  mission  within  a  reasonable  period  (2  to  3  years)  should  be 
terminated  to  develop  new  initiatives    .  ."  (p  32). 

Division  Response:  EEEL  conducts  fundamental  research  as  an  integral  part  of  many  of  its 
measurement-development  projects.  Most  of  the  fundamental  research  projects  that  EEEL 
pursues  are  focused  on  topics  likely  to  have  outcomes  benefiting  measurement  development  for 
U.S.  industry.  That  is,  EEEL  conducts  directed  fundamental  research.  EEEL  does  not  bound  the 
amount  of  directed  fundamental  research  that  it  conducts  to  support  its  measurement  mission    It 
may  also  conduct  some  fundamental  research  that  is  not  directed  toward  potential  outcomes 
benefiting  measurement  development.  The  criteria  for  identifying  a  suitable  project  are  (1) 
unusual  opportunity  for  high  impact,  and  (2)  some  special  reason  for  EEEL  to  be  a  performer. 

1995  Panel  Comment:  The  panel  reemphasizes  the  need  to  evaluate  progress  and  relevance  of  all 
projects  within  the  division  on  an  annual  basis 

1994  Panel  Recommendation:  "The  panel  recommends  that  EEEL  further  develop  its  strategy 
for  project  selection,  considering  published  methodologies  such  as  the  portfolio-based 
methodology  .  .  ."  (p.  32) 

1995  Panel  Comment:    The  division  provided  no  written  response    The  project  selection  criteria 
presented  are  optimized  on  a  project  level,  not  a  portfolio  level    It  is  evident  from  the  division's 
1995  presentation  that  awareness  of  portfolio  management  is  heightened  relative  to  last  year,  but 
a  coherent  response  or  plan  is  apparently  not  forthcoming. 


Electromagnetic  Fields  Division 

Mission 

The  Electromagnetic  Fields  Division  develops  and  evaluates  systems,  devices,  and 
methods  for  measurement  and  analysis  of  radio  frequency  electromagnetic  fields,  signals,  noise, 
interference,  and  properties  of  materials  for  guided  and  freely  propagating  fields,  including 
frequency  and  time  domain  representation  of  electromagnetic  fields  and  their  interaction  with 
materials  and  structures,  provides  essential  measurement  and  calibration  services,  enabling 
industry  and  government  to  solve  important  national,  commercial,  industrial,  and  military 
problems  such  as  evaluating  the  performance  of  microwave  and  millimeter  systems,  components. 
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and  materials  used,  for  example,  in  advanced  radar,  satellite  and  mobile  communications,  and 
automated  test  systems,  assists  other  agencies  with  measurement-related  issues  such  as 
determining  levels  of  nonioniidng  radiation  and  solving  electromagnetic  interference  problems; 
and  disseminates  results  to  industry,  universities,  and  other  government  agencies  to  foster 
effective  research,  development,  manufacturing,  and  marketplace  equity 


Resources 

Fiscal  year  1994  funding  for  the  Electromagnetic  Fields  Division  totaled  $10.3  million,  of 
which  $5.2  million  was  from  STRS,  $0  5  million  fi-om  ATP,  $1.2  million  from  fees  for  calibrations 
and  other  services,  and  $3.4  million  from  OA    Fiscal  year  1995  funding  was  estimated  at  the  time 
of  the  assessment  to  total  $9.5  million:  $3.9  million,  STRS,  $0.4  million,  ATP,  $1.2  million, 
calibrations  and  other  services;  and  $3  9  million,  OA    Of  a  total  staff  of  71,  44  are  technical 
professionals,  and  1 8  hold  a  PhD  degree. 


Assessment  of  Priorities  and  the  Priority-Setting  Process 

The  division's  priority-setting  and  priority-shifting  methodologies  need  improvement 
During  fiscal  year  1994,  the  division  attempted  to  react  to  new  metrology  demands.  Historically, 
this  division  has  been  funded  primarily  by  other  agencies,  principally  the  Department  of  Defense 
(DOD)    With  such  funds  decreasing,  with  only  modest  increases  in  STRS  funding,  and  with 
substantial  calibration  services  requirements,  the  division  has  been  unable  to  readily  shift  priorities 
among  programs  to  meet  new  requirements    One  alternative  is  the  solicitation  of  new  funding 
from  the  NlST-wide  initiative  process,  unfortunately,  the  initiative  process  is  excessively  long. 
The  division  needs  a  more  timely  method  of  evaluating  the  relevance  of  programs,  adding  and 
eliminating  programs,  and  reassigning  and  retraining  personnel    With  a 
3-year  gap  between  the  submittal  of  a  funding  initiative  and  the  receipt  of  the  funding,  it  is 
imperative  that  the  division  anticipate  industry's  need  at  least  4  years  in  advance    Despite  this, 
planning  must  be  completed  quickly  to  give  the  organization  a  clear  set  of  goals  and  plans. 

The  highly  successful  recent  workshop,  "Electromagnetic  Interference  and 
Electromagnetic  Compatibility  (EMI/EMC)  Metrology  Challenges  for  Industry,"  suggests  the 
division  should  develop  an  EMC  initiative  for  fiscal  year  1997,  provide  feedback  to  industry  on 
the  basis  of  the  workshop,  and  hold  additional  workshops  to  provide  further  guidance  as  the 
initiative  develops    One  way  for  NIST  to  help  industry  with  EMC  issues  is  to  become  involved 
with  International  Electrotechnical  Commission  technical  committees    NIST  is  highly  regarded 
internationally  as  an  unbiased  and  competent  organization  and  could  have  great  influence    Many 
metrology  issues  are  involved  in  the  new  electromagnetic  standards  being  developed 
internationally    The  priority  list  developed  at  the  EMI/EMC  workshop  received  widespread 
support,  and  the  division  must  respond  to  these  as  soon  as  possible  or  lose  credibility    The 
division  can  also  act  as  a  catalyst  for  the  EMC  industry,  helping  steer  industrial  work  toward 
desired  results  by  acting  as  expert  consultants  to  form  and  oversee  industry  committees    This 
action  would  receive  leadership  support  from  industry,  as  most  industries  would  be  flattered  to 
say  they  consulted  with  NIST  on  a  particular  issue 
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The  division  has  been  actively  investigating  the  ways  in  which  it  can  contribute  to 
industry's  needs  in  the  rapidly  expanding  field  of  wireless  communications    The  division  should 
consolidate  the  knowledge  gained  and  prepare  an  initiative  on  wireless  as  soon  as  feasible    This 
initiative,  and  the  funding  that  should  result  fi-om  it,  would  strengthen  and  further  encourage 
interaction  among  the  three  groups  that  make  up  the  division,  since  an  initiative  on  wireless  would 
include,  for  example,  microelectronics  metrology  within  the  Microwave  Metrology  Group,  the 
electromagnetic  properties  of  materials  and  antenna  mutual  coupling  effects  within  the  Antenna 
and  Materials  Metrology  Group,  and  a  variety  of  emissions  and  immunity  metrology  issues  within 
the  Fields  and  Interference  Metrology  Group    A  workshop  similar  to  the  EMI/EMC  workshop 
would  be  an  excellent  way  to  obtain  industry  input  on  pressing  needs  and  to  help  set  priorities  for 
the  wireless  area. 

The  division  should  reexamine  the  value  to  its  current  and  future  activities  of  providing 
extensive  routine  calibrations  to  industry    It  should  also  examine  possible  alternative  ways  of 
producing  traceability  to  hOST  and  transfers  to  industry    The  division  should  move,  wherever 
possible,  in  the  direction  of  reducing  routine  calibrations. 


Assessment  of  Technical  Programs 

The  division  has  ongoing  programs  in  power  standards  and  measurements;  impedance, 
attentuation,  and  voltage  standards  and  measurements,  network  analysis  and  measurement; 
metrology  for  microelectronics  (inexpensive  and  accurate  electrical  measurement  methods  for 
high-speed  and  high-frequency  microelectronic  devices),  noise  standards  and  measurements; 
development  of  electromagnetic  reference  fields  and  transfer  probe  standards,  emission  and 
immunity  metrology,  antenna  measurement  theory,  metrology  for  antenna,  radar  cross  section, 
and  space  systems,  and  measurement  of  electromagnetic  properties  of  dielectric  and  magnetic 
materials.  In  addition,  the  division  provides  calibrations  and  special  test  measurements  to  industry 
and  government  laboratories    The  division's  technical  programs  are  generally  good,  and  the 
quality  of  the  work  continues  to  be  of  the  high  standard  that  has  always  been  NIST's  trademark 
The  equipment  and  facilities  are  adequate  for  current  programs  but  may  require  significant 
updating  as  the  division  continues  its  transition  from  OA  support  to  support  by  private  industry. 
The  division  needs  a  roadmapping  process  to  define  these  requirements  in  advance  of  actual  need. 
The  division's  programs  overall  are  effective  in  reaching  goals  and  in  transitioning  technology  to 
industry.  However,  some  efforts  are  not  in  the  direction  needed  by  industry 

The  panel  notes  the  receipt  of  the  prestigious  Institute  of  Electrical  and  Electronics 
Engineers  (IEEE)  Morris  E  Leeds  Award  by  two  members  of  the  division's  technical  staff,  which 
is  a  well-deserved  recognition  for  their  pioneering  and  basic  work  on  microelectronics  metrology. 
The  panel  encourages  further  support  for  fijture  extensions  of  this  activity,  particularly  in 
interconnects  and  package  effects,  and  the  crosstalk  effects  that  can  result 


Assessment  of  Impact  on  Industry 

The  division's  technical  work  has  traditionally  had  a  limited  nondefense  impact,  as  much 
of  this  work  has  been  related  to  DOD  needs    The  division  is  in  a  transition  from  being  heavily 
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DOD  supported  to  being  more  commercially  oriented    Although  the  division's  balance  of 
programs  is  still  weighted  towards  defense,  opportunities  to  support  private  industry  are 
numerous  and  diverse    Customers  of  the  division  could  include  the  commercial  satellite 
communications  industry,  the  telecommunications  industry,  and  the  cable  television  industry    The 
curtent  focus  on  wireless  technology,  personal  communication,  and  information  transfer  should 
provide  a  good  environment  for  making  this  transition.  Many  of  the  division's  skills,  including 
EMC,  antenna  metrology,  and  noise  measurements,  are  very  appropriate  to  these  new 
technologies    As  the  division  shifts  its  programs  to  commercial  applications,  it  must  make  more 
effort  to  ascertain  industry's  critical  needs. 

One  area  where  the  division  does  supply  significant  industrial  support  is  in  calibrations  of 
both  antennas  and  components.  The  availability  of  standards  that  have  known  error  bounds  and 
are  traceable  to  national  standards  has  a  very  positive  impact  on  the  competitiveness  of  U.S. 
industry    A  quality  assurance  process  might  enhance  the  value  of  this  service  to  industry 

The  Antenna  and  Materials  Metrology  Group  is  heavily  dependent  on  OA  funds  and,  as  a 
result,  the  majority  of  its  work  is  DOD  related    However,  the  commercial  satellite  industry  is  very 
strong  and  intensely  competitive.  There  should  be  opportunities  for  NIST  to  provide  metrology 
support  for  this  industry  and  as  a  result  make  U.S.  satellite  manufacturers  more  competitive  in  the 
world  market.  Specific  opportunities  include  more  accurate  gain  measurements,  more  accurate 
polarization  measurements,  and  more  accurate  pattern  measurements.  A  key  contact  would  be 
FNTELSAT,  an  international  organization  that  has  driven  many  of  the  improvements  in  satellite 
antenna  metrology    The  panel  recommends  that  the  division  initiate  contact  with  INTELSAT  to 
define  opportunities  to  improve  the  competitiveness  of  US  manufacturers. 


Recommendations 

The  following  are  the  panel's  recommendations  for  the  Electromagnetic  Fields  Division. 

•  The  Electromagnetic  Fields  Division  should  complete  its  roadmap  exercise  as  promptly  as 
possible  to  clarify  its  goals  and  plans 

•  The  division  should  invest  more  effort  in  ascertaining  industry's  critical  needs  as  it  continues 
its  transition  from  providing  defense-related  to  commercial  support 

•  The  division  should  follow  up  on  its  recent  EMI/EMC  workshop  with  a  fiscal  year  1 997 
initiative  in  EMC 

•  A  workshop  similar  to  the  EMI/EMC  workshop  should  be  held  to  give  direction  to  a  division 
initiative  in  wireless  communications 

•  The  division  should  consider  a  quality  assurance  process  for  its  calibrations  and  other  services 
to  industry. 

•  The  division  should  consider  alternate  means  of  providing  routine  calibration  services  to 
industry. 
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The  division  should  better  familiarize  itself  with  industry's  metrology  requirements  through 
greater  participation  in  international  standards  committees  and  conferences  and  visits  to  US 
industry. 

The  division  should  initiate  contact  with  INTELSAT  to  define  opportunities  to  improve  the 
competitiveness  of  the  US  commercial  satellite  industry. 


Electromagnetic  Fields  Division  Responses  to  Fiscal  Year  1994  Recommendations 

The  panel  finds  the  Electromagnetic  Fields  Division  generally  responsive  to  its  1994 
recommendations,  quoted  fi"om  the  fiscal  year  1994  assessment  report  below.  The  division  should 
be  complimented  for  its  efforts  in  gathering  information  and  data  necessary  to  develop  a 
systematic  roadmap  for  fijture  technology  needs,  for  its  controlled  reduction  and  upgrading  of  the 
staff",  and  for  organizing  an  industry  workshop  on  EMI/EMC  metrology  needs    Of  special  note  is 
the  division's  action  to  expand  work  on  monolithic  microwave  integrated  circuits  (MMICs)  into 
the  Microelectronics  Metrology  project,  which  reflects  increased  emphasis  on  microelectronics 
packaging  and  interconnections 

J 99-4  Panel  Recommendation:  "Commercial  applications  based  on  microwave  fi"equencies  should 
lead  to  a  large  number  of  metrology  issues  involving  microwave  integrated  circuits,  antennas,  and 
electromagnetic  compatibility    The  Electromagnetic  Fields  Division  should  be  positioned  to 
support  US  industry  in  these  emerging  technologies  and  applications,  emphasizing  the 
development  of  low-cost  transfer  standards  and  calibration  techniques  that  are  easily  used  by 
industry"  (pp  35-36). 

Division  Response:  The  division  has  recognized  that  industry  needs  measurement  techniques  that 
are  both  accurate  and  cost  effective  and  has  been  interacting  with  industry  to  develop  and  improve 
measurement  techniques  to  meet  these  criteria    In  support  of  MMICs,  the  division  has  developed 
calibration  techniques  that  are  based  on  compact  calibration  sets  that  use  imperfect  resistors  and 
lines  typical  of  commercial  MMIC  devices    The  division  continues  to  distribute  user-fiiendly  on- 
wafer  calibration  and  measurement  software  and  has  enhanced  it  to  ensure  compatibility  with  a 
wider  range  of  commonly  used  instrumentation    The  division  is  working  on  alternative  methods 
that  will  reduce  time  and  expense  for  providing  electrical  traceability  for  S-parameter 
measurements  in  both  coaxial  and  waveguide  transmission  lines.  With  respect  to  antenna 
measurements,  the  division  is  evaluating  a  self-calibration/mirror  image  technique  that  could 
reduce  costs  by  a  factor  of  two  for  calibrating  low-cost  transfer  probe  and  horn  standards  and  is 
investigating  an  infi-ared  holographic  method  for  the  measurement  of  antennas  that  shows  promise 
for  rapidly  collecting  planar  near-field  data  and  for  on-line  diagnostic  testing  in  an  assembly  line 
environment. 

J994  Panel  Recommendation:  "The  division  should  consult  with  relevant  industrial  organizations 
to  assess  what  new  measurement  problems  and  needs  for  new  standards  are  expected  to  arise 
fi"om  new  microwave  applications  and  then  prepare  a  systematic  roadmap  and  develop  appropriate 
initiatives.  .  .  ."  (p.  36). 
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Division  Response:  The  division's  staff  members  are  continually  seeking  input  as  they  interact 
with  representatives  of  industry  at  conferences  and  in  standards  activities  and  as  they  visit 
individual  laboratories.  The  division  has  also  participated  in  formal  workshops  and  planned  others 
such  as  the  EMC/EMI  workshop    Each  group  is  developing  a  long-range  plan  for  all  its  technical 
areas,  providing  references  for  industrial  needs,  and  prioritizing  each  area    This  information  will 
then  be  combined  to  form  the  division  roadmap.  A  microwave  element  responding  to  new 
applications  in  industry  is  being  considered  in  fiscal  year  1997  initiative  development 

J99-i  Panel  Recommendation:  "Vehicle  collision-avoidance  systems  operating  at  millimeter 
wavelengths  and  extensively  utilizing  microwave  integrated  circuits  will  be  on  the  market  in  a  few 
years    Since  the  division  has  the  special  calibration  knowledge  and  skills  required  to  solve  issues 
arising  from  these  applications,  the  division  should  survey  manufacturers  to  determine  NIST's 
support  and  develop  the  appropriate  initiatives"  (p.  36) 

Division  Response:  The  division  is  measuring  and  analyzing  60  GHz  planar  near-field  data  on  a 
prototype  subarray  that  will  be  used  as  an  element  for  millimeter-wave  arrays.  This  project 
includes  both  theoretical  analysis  and  implementation  and  is  expected  to  show  that  the  probe- 
position  error  correction  methods  developed  for  these  higher-frequency  measurements  are  sound. 
A  proposal  to  investigate  terahertz  frequency  metrology  methods  and  devices  has  been  prepared 
and  is  being  considered  in  fiscal  year  1997  initiative  development. 

1994  Panel  Recommendation:  "Many  different  types  of  antennas  will  be  required  for  new 
commercial  applications,  but  in  some  cases  ...  a  proliferation  of  communications  and  radar 
functions  will  be  present  simultaneously    Antennas  of  different  types  operating  at  different 
frequencies  will  lead  to  an  unprecedented  number  of  mutual  coupling  effects    The  division  should 
be  sensitive  to  new  technical  issues  arising  as  a  result  and  consider  initiating  new  measurement 
programs  to  identify  and  help  solve  such  problems"  (p.  36). 

Division  Response:  The  division  has  done  work  in  the  past  on  modeling  mutual  coupling  between 
co-sited  antennas,  and  much  of  that  work  will  apply  in  this  application.  The  division  is  also 
studying  coupling  in  lossy  multimode  transmission  line  systems  that  will  apply  to  the  integrated 
circuits  and  transmission  lines  used  in  these  systems    An  equivalent  circuit  theory  applicable  to 
multimode  systems  has  been  developed  and  mask  sets  with  test  structures,  laid  out.  Fabrication  of 
structures  is  planned  for  fiscal  year  1995. 

1994  Panel  Recommendation:  "New  commercial  applications  require  improved  metrology  in 
electromagnetic  compatibility  involving  measurements  for  both  immunity  and  emissions.  The 
division  has  not  developed  a  systematic  roadmap  for  these  issues  based  on  its  own  technical 
judgment  and  surveys  of  industrial  needs    Such  a  strategic  planning  approach  should  be 
supported  by  appropriate  levels  of  funding  and  staff'  (p.  36). 

Division  Response:  The  division  has  conducted  extensive  industrial  surveys  to  identify  first-hand, 
up-to-date  information  on  the  measurement-related  needs  and  problems  associated  with 
electromagnetic  compatibility  This  survey  identified  about  50  technical  issues  that  U.S.  industries 
face  as  they  compete  in  the  global  marketplace  and  advancements  needed  in  EMI/EMC  metrology 
research  In  January  1995,  the  division  conducted  a  workshop  entitled  "EMI/EMC  Metrology 
Challenges  for  Industry"  to  prioritize  the  advances  needed  in  EMI/EMC  metrology  and  develop  a 
systematic  roadmap. 
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1994  Panel  Recommendation:  "The  division  should  provide  leadership  in  electromagnetic 
compatibility,  since  there  is  no  industrial  organization  representing  that  area    In  this  context,  the 
division  should  be  proactive  in  helping  to  prepare  US.  industry  to  meet  the  new  European  Union 
EMC  standards,  including  standards  harmonization,  EMC  metrology,  and  EMC  calibration 
standards"  (p  36) 

Division  Response:  The  division  actively  participates  in  many  standards  committees  in  the  IEEE, 
American  National  Standards  Institute  (ANSI),  Society  of  Automotive  Engineers,  and  American 
Society  for  Testing  and  Materials  Group  members  have  played  a  major  role  in  developing  many 
cost-effective  EMC  standards  Some  of  the  EMC  standards  adapted  by  ANSI  were  proposed  to 
the  International  Special  Committee  on  Radio  Interference  as  international  standards 

J994  Panel  Recommendation:  "The  division  should  determine  the  support  it  can  provide  to  the 
information  superhighway  on  issues  such  as  EMC,  antennas,  microwave  integrated  circuits,  and 
other  technologies"  (p  36) 

Division  Response:  The  division  has  received  funding  fi-om  the  Nil  initiative  to  support  work  in 
characterizing  interconnections    The  division  is  applying  its  expertise  in  on-wafer  calibration  to 
the  development  of  calibration  techniques  for  the  time  domain  systems  most  commonly  used  in 
testing  these  information  networks    The  division  is  also  pursuing  other  work  associated  with  the 
wireless  aspects  of  this  technology. 

1994  Panel  Recommendation:  "The  division  should  consider  retraining  existing  staff  and  hiring 
new  staff,  particularly  younger  staff,  to  address  new  applications  There  should  be  some  overlap 
of  new  staff  wath  retirees  to  provide  continuity  in  key  technologies"  (p.  37) 
Division  Response:  The  division  has  begun  the  process  of  retraining,  has  reassigned  staff  to  the 
Noise  project  and  the  Microelectronics  Metrology  Program,  and  anticipates  more  reassignments. 
The  division  is  also  considering  hiring  a  new  person  to  assist  in  the  Microelectronics  project  if 
funding  continues. 

1994  Panel  Recommendation:  "In  anticipation  of  new  increases  in  STRS  funding  and  OA 
funding  decreases,  the  division  should  systematically  examine  which  programs  will  continue  to 
increase  and  which  should  be  phased  out  or  reduce  in  scope        "  (p  37). 

Division  Response:  The  division  is  carrying  out  such  an  analysis  as  one  component  of  long-range 
plan  and  roadmap  development. 


Electromagnetic  Technology  Division 
Mission 

The  Electromagnetic  Technology  Division  develops  and  promotes  advanced  physical 
standards  and  measurement  methods  for  the  magnetics,  electronics,  and  superconductor  industries 
and  their  related  scientific  communities;  employs  phenomena  based  on  magnetics, 
superconductivity,  and  cryoelectronics  to  create  new  measurement  technology  and  associated 
standards  and  apparatus;  advances  the  state  of  the  art  by  basic  research  and  development  of 
requisite  materials,  fabrication  techniques,  instrumentation,  underlying  theory,  and  data  for 
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metrology;  provides  new  measurement  methods,  instrumentation,  imaging  and  characterization 
tools,  and  standards  in  support  of  the  magnetics  industry;  develops  measurement  technology  to 
determine  basic  properties  of  magnetic  materials  and  structures  with  support  from  theoretical 
studies  and  modeling,  collaborates  with  the  magnetic  recording  industry  in  development  of 
metrology  to  support  future  recording  heads  and  media  with  their  ever-increasing  data  density; 
uses  the  unique  properties  of  superconductors  and  cryogenic  electronics  to  invent  and  improve 
measurement  methods  for  electromagnetic  signals  ranging  from  static  voltages  and  magnetic 
fields,  through  audio,  microwave,  infrared,  visible,  and  x-ray  frequencies,  leads  the  international 
community  in  setting  standards  for  measurement  of  superconductor  parameters;  and  provides  the 
metrology  infrastructure  needed  for  the  industrial  development  and  application  of 
superconductors,  both  large  and  small  scale. 


Resources 

Fiscal  year  1994  funding  for  the  Electromagnetic  Technology  Division  was  $6.92  million, 
of  which  $4.88  million  was  from  SIRS,  $0.64  million  from  ATP,  and  $1.4  million  from  OA. 
Fiscal  year  1995  funding  was  estimated  at  the  time  of  the  assessment  to  total  $6.91  million:  $4  89 
million,  STRS,  $0.45  million,  ATP,  and  $1.57  million,  OA.  Of  a  total  paid  staff  of  49,  40  are 
technical  professionals  and  26  hold  a  PhD  degree. 


Assessment  of  Priorities  and  the  Priority-Setting  Process 

The  division  appears  to  be  setting  priorities  effectively.  Magnetics  projects  have  been 
selected  on  the  basis  of  the  technical  needs  and  requirements  of  the  current  magnetics  technology, 
mission,  and  capabilities  and  skills  of  available  personnel.  Ongoing  dialogs  between  division  staff 
and  industrial  and  university  colleagues  have  in  many  cases  led  to  projects  that  were  selected  for 
their  high  potential  impact  and  relevance  to  industrial  and  scientific  needs.  During  the  past  year, 
the  magnetics  program  has  been  built  up  by  redirecting  some  staff  members.  Assignment  of 
priorities  in  the  development  of  low-temperature  instrumentation  is  based  on  standards  activities. 
This  work  is  essential  and  central  to  NIST's  mission.  The  single-electron  tunneling  research  at 
Boulder,  which  could  lead  to  a  quantum  standard  for  capacitance,  is  a  particular  example  of  an 
effort  that  should  move  forward  under  stable  fianding.  Work  on  superconductor  standards 
development  and  representation  of  U.S.  interests  in  international  standards  activities  are  of  ever- 
increasing  importance,  but  the  present  level  of  staffing  in  this  area  is  not  representative  of  its 
potential  impact. 

There  is  a  particular  challenge  associated  with  priority-setting  for  projects  supporting  the 
magnetics  and  superconductivity  industry    These  industries  are  rapidly  moving  and  highly 
competitive,  and  much  of  the  key  technology  developed  in  industrial  laboratories  is  retained  as 
proprietary.  Inadvertent  duplication  of  this  work  at  NIST  diverts  scarce  resources  from 
understaffed  projects.  Particular  attention  to  this  is  necessary  in  the  areas  of  giant 
magnetoresistive  heads,  pirming  in  alloy  superconductors,  and  superconductor  interfaces  in  thin 
films 
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The  magnetics  program  should  be  part  of  a  larger  EEEL-  or  NIST-wide  recognition  of  the 
importance  of  information  storage  (magnetic,  optical,  semiconducting,  ferroelectric,  and  so  on)  as 
an  industry.  There  does  not  seem  to  be  a  process  in  place  to  encourage  the  formation  of  cross- 
divisional  teams  to  identify  and  address  the  needs  of  this  industry. 

The  panel  is  concerned  over  the  laboratory's  lack  of  recognition  of  the  importance  of  low- 
temperature  measurements  as  a  priority    It  has  been  clearly  demonstrated  over  the  past  20  years 
that  many  electrical  measurements  are  best  made  at  low  temperatures    The  collective  impact  on 
the  capabilities  of  NIST  in  the  metrology  and  standards  arenas  has  been  vastly  enhanced  by  the 
low-temperature  research  of  this  division  at  both  Boulder  and  Gaithersburg.  A  deliberate  decision 
should  be  made  at  the  laboratory  level  to  increase,  maintain,  or  shrink  these  activities  to  an 
appropriate  size.  The  present  situation,  in  which  the  budget  of  the  Boulder  group  appears  to  be 
frozen,  resulting  in  the  loss  of  one  staff  member  per  year,  is  demoralizing  a  division  that  is  a 
unique  national  asset. 


Assessment  of  Technical  Programs 

In  most  cases,  the  current  work  within  the  division's  projects  is  at  the  leading  edge  of  such 
research  worldwide  and,  in  some  cases,  has  led  all  such  efforts    Equipment  and  facilities  appear 
to  be  mostly  adequate,  but  if  projects  are  maintained  at  their  present  size,  more  space  should  be 
obtained. 


The  addition  of  a  disk  recording  test  stand  and  vibrating  sample  magnetometer  would  add 
substantial  value  to  the  division's  program  by  allowing  direct  recording  performance  of 
components  such  as  novel  heads  and  disks  of  different  microstructure  to  be  quantified 

The  panel  notes  with  concern  the  growing  use  of  postdoctoral  personnel  in  positions  that 
are  in  the  critical  path  of  important  projects  in  magnetics  and  low-temperature  measurements    If 
these  temporary  personnel  are  not  retained,  their  knowledge  will  be  lost  and  the  investment 
required  to  create  that  knowledge  will  be  wasted    Another  unintended  effect  of  the  continued 
hiring  of  postdoctoral  personnel  is  an  imbalance  between  the  number  of  scientists  and  engineers  in 
the  division,  since  engineers  do  not  traditionally  seek  postdoctoral  positions    Many  efforts,  such 
as  the  high-resolution  x-ray  detector  and  the  scanning  micromagnetic  recording  system,  would 
benefit  from  the  presence  of  more  hands-on  development  engineers    As  hiring  opportunities 
become  available,  consideration  should  be  given  to  supplementing  the  division's  traditionally 
strong  physics  base  with  additional  graduate-level  electrical  engineers 

If  an  assistant  division  chief  is  added,  the  panel  suggests  that  the  management  team  should 
be  balanced  by  a  person  with  engineering  experience  either  in  magnetics  or,  more  broadly,  in 
information  storage.  Such  a  person  would  allow  the  current  magnetics  group  to  take  leadership  in 
defining  NIST's  relationship  to  the  information  storage  industry. 

An  office  in  information  storage,  including  magnetics,  similar  to  the  Office  of 
Microelectronics  Programs,  could  be  part  of  the  responsibilities  of  a  deputy  chief  This  office 
could  serve  to  maintain  heightened  awareness  within  NTST  of  storage  industry  technologies  and 
roadmaps  and  general  awareness  of  market-related  requirements  in  the  information  processing 
and  storage  industries,  and  it  could  help  to  organize  meetings,  as  appropriate,  to  examine 
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additional  areas  of  opportunity  in  magnetics  and  storage,  such  as  perpendicular  recording,  contact 
recording,  or  magnetic  studies  for  the  health  care  industry  (recently  identified  by  NIST  as  an 
umbrella  initiative  area  for  fiscal  year  1997). 

The  project  management  structure  used  in  this  division,  without  group  leaders,  seems  to 
be  woricing  well  and  is  consistent  with  the  modem  industry  practice  of  eliminating  middle  levels  of 
management.  It  has  the  advantage  of  allowing  senior  technical  people  more  involvement  in  the 
decision-making  process,  while  still  giving  them  enough  time  to  maintain  their  unique  technical 
skills  at  the  laboratory  bench 

With  the  reorganization  of  the  Electromagnetic  Technology  Division,  EEEL  has  made  a 
clear  statement  that  every  effort  will  be  made  to  enhance  and  enlarge  the  magnetic  and  storage 
programs.  This  is  evidenced  by  growth  for  magnetic  storage  in  the  fiscal  year  1997  budget  plan; 
interaction  with  industrial  storage  standards  and  technology  organizations,  such  as  the 
International  Disk  Equipment  and  Materials  Association  (IDEMA)  and  the  National  Storage 
Industry  Consortium  (NSIC);  the  plan  for  the  division  to  be  the  primary  focus  of  the  NIST 
initiative  for  magnetic  storage,  and  its  interactions  through  the  NIST  Industry  Fellows  Program 
with  companies  such  as  the  Maxtor  Corporation  (Longmont,  Colo.). 

The  quality  of  the  magnetics  projects  in  the  Electromagnetic  Technology  Division  is  high, 
as  evidenced  by  the  number  of  ongoing  collaborations  with  industrial,  governmental,  and 
university  counterparts  (for  example,  IBM,  Quantum,  Hewlett-Packard,  Censtor,  Micropolis, 
Maxtor,  NSIC,  Stanford,  University  of  California  at  San  Diego),  and  the  external  visibility 
received  by  NIST  scientists  at  external  conferences  and  meetings.  Developments  of  particular 
interest  include  the  scanning  near-field  optical  microscope,  which  has  the  potential  for 
revolutionizing  the  ease  with  which  high-resolution  magnetics  measurements  are  made,  the 
scanning  magnetic  recording  system,  which  could  potentially  find  broad  application  as  a  research, 
development,  and  eventually  a  manufacturing  tool,  high-resolution  scanning  microscopy  for  head 
and  media  applications,  and  the  correlation  of  microstructure  with  magnetic  performance  and 
deposition  conditions    The  division  should  continue  to  develop  magnetic  microscopy  tools  with 
detailed  comparison  with  theoretical  modeling  calculations    Successfijl  development  of  these 
tools  could  have  broad  application  across  the  industry.  In  general,  the  equipment  for  the 
magnetics  projects  is  sufficient 

In  the  Superconductor  Standards  and  Technology  project,  the  interiaboratory  comparisons 
of  critical  current  (7^)  and  the  development  of  7^  simulators  are  central  to  the  orderly  growth  of 
the  superconductor  industry.  The  division's  work  in  these  areas  is  of  high  quality  and  is 
appropriate    There  curtently  exists  no  coherent  US  industrial  leadership  in  the  development  of 
international  standards  for  commercial  superconductors,  and  NIST  must  assume  leadership  in  this 
area.  Unfortunately,  the  present  level  of  divisional  staffing  and  support  is  insufficient  to  compete 
effectively  with  a  large  Japanese  effort    If  this  level  is  not  supplemented  in  the  next  year,  the 
result  may  be  Japanese  domination  of  the  international  standards  in  this  critical  area. 
Supplementation  should  include  adding  staff  dedicated  to  this  effort  and  providing  financial  and 
organizational  support  and  involvement  of  representatives  of  industry  and  other  government 
agencies  such  as  DOD  and  DOE    Work  on  artificial  pinning  centers  in  NbTi,  although  of 
excellent  technical  quality,  falls  outside  the  NIST  mission  and  should  not  be  continued  past  fiscal 
year  1995. 

In  the  Superconductor  Interfaces  and  Electrical  Transport  project,  determination  of  the 
effect  of  strain  on  critical  curtent  in  bulk  superconductors  is  an  area  in  which  the  division  has 
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historically  excelled,  and  this  work  should  continue.  Particular  focus  on  rapid  evaluation  of  high- 
temperature  superconducting  (HTS)  materials  and  devices  is  essential  to  continued  progress  in 
conductor  development. 

Research  on  interfaces  in  bulk  superconductors  should  focus  on  high-current,  low- 
resistance  contacts  to  HTS  wires  and  on  the  development  of  superconducting  splices  between 
such  wires.  The  division  is  uniquely  qualified  to  address  these  areas,  and  success  would  enable  a 
number  of  applications  both  in  instruments  and  large-scale  devices.  An  initiative  on  interfaces  of 
thin  film  HTS  materials  has  recently  been  funded  by  the  Office  of  Naval  Research  and  the 
National  Science  Foundation  at  a  number  of  institutions,  and  a  body  of  proprietary  knowledge  in 
this  area  exists  in  U.S.  industry,  so  care  must  be  taken  to  avoid  duplication  of  these  efforts. 

The  measurement  of  ac  loss  in  superconductors  and  the  development  of  instrumentation  to 
accomplish  this  in  the  Magnetic  Instruments  and  Materials  Characterization  project  is  an 
appropriate  and  timely  activity    However,  the  diversion  of  personnel  from  this  effort  to  magnetics 
has  postponed  construction  of  the  flux  integration  magnetometer  designed  for  these 
measurements    Given  the  rapid  evolution  of  superconductors  designed  for  reduced  loss,  this 
delay  has  had  an  immediate  effect  on  the  industry. 

The  Nanoscale  Cryoelectronics  project,  the  Josephson  Array  Development  project,  the 
High-7"c  (critical  temperature)  Electronics  project,  and  the  High-Performance  Sensors,  Infrared 
Deteaors,  and  Mixers  project  represent  the  invention,  creation,  and  development  of  the  devices 
that  make  measurements  at  low  temperatures  first  possible  and  then  more  accurate.  This  work  is 
of  the  highest  quality    These  projects  are  staffed  by  a  unique  assembly  of  highly  talented 
individuals  which,  as  a  team,  represents  an  asset  for  NIST  and  for  the  nation    Simply  stated,  there 
is  not  a  comparable  center  of  excellence  elsewhere,  either  in  a  university,  industry,  or  other  federal 
laboratory    This  center  of  excellence  has  had  great  impact  on  the  mission,  capabilities,  and 
reputation  of  NIST  itself 


Assessment  of  Impact  on  Industry 

NIST's  traditional  impact  on  industry  has  been  as  a  provider  of  fundamental  standards.  In 
magnetics,  the  division's  program  is  below  critical  mass  and  has  only  limited  impact    In 
superconductivity,  however,  the  division  provides  basic  low-temperature  metrology  which 
underiies  the  needed  fundamental  standards  for  this  area  of  technology    As  a  result,  the  work  of 
the  division  has  significant  indirect  impact  on  industry.  The  impact  of  this  division  in  large-scale 
superconductivity  is  greatest  in  those  areas  where  NIST  provides  unique  facilities  or  measurement 
capabilities  and  furnishes  timely  feedback  to  industry    Examples  are  work  on  development  of 
standards  for  commercial  superconductors  and  measurements  of  the  effects  of  strain  on  /^ 

Projects  focused  on  low-temperature  measurements  have  had  great  industrial  impact.  This 
ranges  from  the  transfer  of  the  voltage  standard  for  manufacture  by  a  small  US  company  to 
many  valuable  one-on-one  collaborations    The  value  of  the  former  is  easy  to  measure,  but  the 
latter  is  also  significant  and  important 

It  is  clear  that  the  US  superconductor  industry,  previously  based  on  low-temperature 
superconducting  materials,  is  begirming  to  expand  because  of  recent  discoveries  of  high- 
temperature  superconducting  materials    Four  new  high-temperature  superconductor  companies 
employ  over  300  people  and  continue  to  expand    Their  technology  is  advanced  enough  to 
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develop  some  products  but  remains  fragile  and  immature  As  the  industry  continues  to  grow,  it 
will  be  increasingly  important  for  the  division  to  take  on  the  role  that  other  EEEL  divisions  play 
for  more  mature  industries  such  as  semiconductors,  by  providing  standards,  measurements,  and 
leadership  in  international  standard  setting. 


Recomnnendations 

The  following  are  the  panel's  recommendations  for  the  Electromagnetic  Technology 
Division 

•  The  addition  of  a  disk  recording  test  stand  and  a  vibrating  sample  magnetometer  would  add 
substantial  value  to  the  Electromagnetic  Technology  Division's  program  by  allowing  direct 
recording  performance  of  components  to  be  quantified 

•  EEEL  should  consider  an  office  in  information  storage,  similar  to  the  Office  of 
Microelectronics  Programs,  to  leverage  NIST  programs  in  service  to  the  storage  industry. 

•  A  cross-divisional  or  perhaps  cross-laboratory  initiative  should  be  considered  to  explore  and 
address  the  needs  of  the  information  storage  industry 

•  The  present  magnetics  projects  should  be  recognized  as  a  priority  for  that  industry,  and  should 
be  staffed  and  financed  appropriately. 

•  Work  on  artificial  pinning  centers  in  NbTi  falls  outside  the  NIST  mission  and  should  not  be 
continued  past  fiscal  year  1995 

•  The  importance  of  low-temperature  measurement  techniques  should  be  recognized  and  a 
staffing  and  budget  plan  be  put  in  place  to  maintain  the  national  asset  represented  by  these 
projects  at  Gaithersburg  and  Boulder. 

•  The  program  to  develop  a  magnetometer  for  the  measurement  of  ac  losses  in  superconductors 
either  should  be  abandoned  or  should  be  pursued  with  sufficient  focus  to  ensure  timely 
completion  of  the  device  and  the  availability  of  staff  to  operate  it. 


Electromagnetic  Technology  Division  Responses  to  Fiscal  Year  1994  Recommendations 

Below  are  the  panel's  1994  recommendations  for  the  Electromagnetic  Technology 
Division,  quoted  from  the  fiscal  year  1994  assessment  report,  followed  by  the  division's  response. 

1994  Panel  Recommendation:  "The  .      [division]  should  have  one  mission  and  strategy 
statement  that  is  well  understood  by  all  staff  This  should  focus  work  on  fewer,  key  projects,  and 
lead  to  stable  staffing  for  those  projects   Resources  in  superconductivity  should  be  limited  . .  ." 
(P  46). 
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Division  Response:  At  the  time  of  the  fiscal  year  1994  panel  meeting,  the  separation  of  the 
Optoelectronics  Division  fi-om  the  Electromagnetic  Technology  Division  was  still  in  the  planning 
phase    The  restructuring  of  the  new  division  has  proceeded  over  the  year,  although  progress  has 
been  slower  than  originally  anticipated    The  restructuring  has  concentrated  on  a  number  of  key 
projects  with  strong  researcher/project  leader  management    A  new  mission  statement  has  been 
prepared  which  recognizes  an  increased  commitment  to  providing  support  to  the  magnetics 
industry  while  maintaining  worid  leadership  in  selected  aspects  of  superconductivity 

1994  Panel  Recommendation:  "The  single-electron  tunneling  activity  needs  to  switch  from 
competence-building  funding  to  stable  core  funding    Funding  and  staffing  for  this  project  should 
be  stabilized  at  an  adequate  level  and  this  project  made  a  core  component  of  any  NlST-wide 
activity  in  this  area"  (p  47). 

Division  Response:  The  SET  activity  was  begun  in  1987.  The  importance  of  the  work  was 
recognized  with  competence  funding,  to  continue  through  this  fiscal  year.  A  complementary 
program  was  begun  in  1992  by  the  Electricity  Division    That  project  is  also  competence  flinded, 
with  funding  through  1998    Researchers  in  the  two  divisions  have  established  a  working 
relationship  that  both  division  chiefs  regard  as  excellent 

The  restructured  Electromagnetic  Technology  Division  is  broadening  the  focus  of  its  work 
with  a  new  project  called  Nanoscale  Cryoelectronics  to  develop  novel  integrated  circuits  for 
metrology  based  on  the  unique  properties  of  submicrometer  electronic  devices  operated  at 
temperatures  below  1  K    The  project  has  been  augmented  with  a  National  Research  Council 
postdoctoral  research  fellow,  who  will  focus  on  the  development  of  SET-based  metrology, 
including  prospective  capacitance  standards    The  division  plan  calls  for  hiring  a  second 
postdoctoral  researcher  to  work  on  a  novel  x-ray  detector    The  division  is  also  seeking  to  hire  a 
researcher  of  world  renown  for  the  general  area  of  nanoscale  cryoelectronics.  The  division 
recognizes  that  identifying  funding  sources  for  the  standards  applications  of  SET  work  is 
important  and  that  this  focus  of  the  work  needs  to  be  nurtured 

1994  Panel  Recommendation:  "With  the  development  of  superconducting  materials  and 
instruments  that  operate  without  a  liquid  cryogen  .  .    there  is  an  enormous  need  for  techniques 
and  procedures  to  give  quality  assurance  in  the  10  to  77  K  range  It  is  important  that  EEEL 

anticipate  and  adequately  support  the  needs  of  the  new  division  in  this  area"  (p  47) 
Division  Response:  All  of  the  project  areas  involved  with  large-scale  applications  of 
superconductors  are  either  using  or  building  systems  for  measurement  studies  at  temperatures 
above  4  K    Specifically,  instrumentation  and  temperature-variable  systems  for  developing  critical 
current  standards,  measuring  field-orientation  dependence  of  7^,  determining  ac  loss,  and 
measuring  strain  effects  in  coils  are  all  either  in  place  or  will  be  in  fiscal  year  1995    The  NIST 
superconductor  simulator,  which  enables  the  testing  of  instrumentation  apart  from  the  cryogenic 
system,  is  an  important  tool  for  the  development  of  these  systems  by  industry.  EEEL's  role  as  a 
leader  in  the  international  standards  community  in  this  field  ensures  involvement  in  associated 
standards  development. 

1994  Panel  Recommendation:  "Work  on  high-7"c  materials  and  processing  should  be  focused  on 
specific  processes  and  device  structures  related  to  the  division's  core  mission    More  generic 
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activity  should  be  deemphasized,  since  it  is  less  likely  to  have  an  impact  and  will  greatly  dilute 
resources"  (p  47). 

Division  Response:  The  focus  in  this  program  areas  is  changing  along  the  lines  suggested    The 
high-temperature  superconductor  junction  work  emphasizes  application  to  voltage  standards,  and 
the  bolometer  work  focuses  on  radiometer  applications  in  collaboration  with  the  Physics 
Laboratory    The  microwave  applications  program  has  ended  its  work  in  tunable  devices  and,  at 
the  request  of  the  industry,  is  starting  a  collaborative  program  on  the  development  of  metrology 
leading  to  improved  devices    Measurement  methods  for  interface  resistance  continue  to  be 
improved,  and  two  new  industry  collaborators  will  be  added  to  the  effort  in  fiscal  year  1995 

J994  Panel  Recommendation:  "The  division  should  obtain  a  sample  preparation  apparatus,  or 
convert  an  existing  one,  for  dedication  to  magnetics"  (p  47). 

Division  Response:  Purchase  of  a  deposition  chamber  for  magnetics  was  initiated  even  before 
this  recommendation  The  new  apparatus  will  have  extensive  analysis  capability,  including  an 
ultra-high-vacuum  scanning  tunneling  microscope  Many  parts  have  been  received  Assembly 
will  take  place  shortly  in  a  newly  remodeled  laboratory. 

1994  Panel  Recommendation:  "The  division  should  decide  if  storage-oriented  work  should  be 
expanded  according  to  its  fiscal  year  1992  plans,  if  not,  it  should  be  discontinued.    .  "  (p  47). 
Division  Response:  Work  is  expanding  at  a  reasonable  pace  given  the  realities  of  resources  and 
facilities.  The  NIST/EEEL  reputation  in  magnetics  research  and  metrology  grew  significantly 
over  the  past  year,  and  many  companies,  both  large  and  small,  requested  either  research 
collaborations  or  assistance  with  measurement  problems    The  program  in  magnetic  recording 
metrology  developed  with  a  focus  on  recording  heads,  especially  magnetoresistive  and  giant 
magnetoresistive  heads    A  series  of  informal  seminars  was  held  with  speakers  fi^om  industry  and 
organizations  such  as  university  recording  centers    In  addition,  EEEL  initiated  a  joint  seminar 
with  the  Physics  Laboratory  in  Gaithersburg  using  EEEL's  new  teleconferencing  facility    EEEL 
initiated  an  interaction  with  IDEMA  and  is  discussing  possible  approaches  to  participation  in  its 
standards  efforts    ANSI  protocol  standards  remain  in  the  domain  of  the  Computer  Systems 
Laboratory  and  are  probably  not  now  appropriate  for  EEEL  participation    EEEL  maintains  a 
strong  interaction  with  the  ATP-funded  project  on  recording  heads  at  the  National  Storage 
Industry  Consortium  and  has  interacted  with  the  ATP  awardee  Nonvolatile  Electronics 
(Plymouth,  Minn.)  in  the  imaging  of  memory  systems. 


Optoelectronics  Division 

Mission 

The  mission  of  the  Optoelectronics  Division  is  to  provide  the  optoelectronics  industry  and 
its  suppliers  and  customers  with  comprehensive  and  technically  advanced  measurement 
capabilities,  standards,  and  traceability  to  those  standards 
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Resources 


The  fiscal  year  1995  funding  of  the  Optoelectronics  Division  was  estimated  at  the  time  of 
the  assessment  to  total  $7  0  million:  $5  7  million  firom  STRS,  $0  1  million  from  ATP,  $0  1 
million  from  fees  for  calibrations  and  other  services,  and  $11  million  from  OA   Of  41  staff 
members,  32  are  technical  professionals,  and  20  hold  a  PhD  degree 


Assessment  of  Priorities  and  the  Priority-Setting  Process 

The  priority  setting  and  project  selection  at  the  division  level  follow  a  somewhat  informal 
but  nevertheless  correct  approach    Not  only  are  current  customer  needs  assessed,  but  these 
assessments  are  also  validated  by  direct  feedback  from  representatives  in  the  specific  fields  where 
the  demand  for  services  is  generated    In  addition,  future  needs  are  anticipated  by  studying 
industry  trends,  resulting  in  a  good  balance  between  projects  for  mature  technologies  and  projects 
for  emerging  technologies 

In  1  year,  the  division  has  gone  from  inception  to  being  a  fully  operational  entity.  This  is  a 
great  accomplishment  and  an  example  of  a  degree  of  responsiveness  not  commonly  found  in  a 
govenunent  agency.  The  panel  congratulates  EEEL  management  and  all  others  who  helped  to 
make  this  transition  possible    During  fiscal  year  1994,  this  new  division  redirected  a  number  of 
ongoing  activities  and  launched  new  thrusts  that  are  well  aligned  with  industrial  needs    The 
division  is  sharpening  its  focus  on  metrology  and  is  doing  excellent  work 

The  optoelectronics  industry  is  growing  rapidly  and  is  the  key  leverage  point  for  future 
information  transport    The  division  will  have  many  challenges  and  opportunities  to  assist  industry 
in  the  coming  decade    The  division  needs  to  maintain  each  of  its  efforts  at  critical  mass  wisely 
and,  in  a  timely  manner,  transition  to  new  important  projects,  such  as  erbium  doped  fiber  amplifier 
(EDFA)  characterization,  and  seek  additional  resources,  both  financial  and  personnel,  to  tackle 
other  aspects  of  photonics  that  cannot  be  pursued  at  present 

The  division's  process  of  holding  quarterly  R&D  reviews  should  yield  good  results  in 
terms  of  program  focus  and  in  effectively  reallocating  resources  throughout  the  year  as  necessary 
to  optimize  the  division's  achievements 

The  division  should  consider  programs  in  optical  storage  and  display  technology, 
photonics  packaging  (including  materials),  and  in  situ  measurements  of  optical  growth  of  new 
devices  (for  example,  blue  lasers)    The  Optoelectronics  Division  should  take  the  lead  in  defining 
and  launching  programs  in  these  areas  and  coordinate  with  other  divisions  as  appropriate  to  serve 
as  the  main  NIST  point  of  contact  for  this  industry 


Assessment  of  Technical  Programs 

The  most  notable  achievement  in  the  Source  and  Detector  Measurement  project  is  the 
characterization  of  photodetector  frequency  response  out  to  60  GHz  to  high  accuracy    This  effort 
was  cross-referenced  to  work  at  the  UK  National  Physical  Laboratory  and  is  representative  of 
the  type  of  collaborative  metrology  work  the  panel  believes  serves  industry  well.  Efforts  continue 
on  power  meter  calibration  and  optical  power  measurement    The  panel  applauds  new  efforts  in 
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source  charaaerization,  such  as  work  on  vertical  cavity  surface  emitting  lasers.  This  work  should 
lead  to  important  metrology  for  advanced  semiconductor  lasers. 

Metrology  efforts  within  the  Fiber  and  Integrated  Optics  project  are  highlighted  by 
development  of  SRMs  for  fiber  cladding  diameter  measurement  and  the  completion  of  an 
international  round-robin  on  measurement  methods  for  fiber  geometry.  This  has  benefited  U.S. 
fiber  manufacturers  by  enabling  them  to  determine  the  accuracy  of  their  own  measurement 
techniques    Standard  Reference  Materials  for  fiber  coating  diameter  measurements  and  ferrule 
size  are  also  under  development,  these  efforts  should  be  equally  beneficial  to  industry    Efforts  to 
develop  and  characterize  SRM  fibers  for  measurements  of  chromatic  dispersion  and  polarization 
mode  dispersion  are  being  expanded    A  new  international  round-robin  on  characterization  of 
EDFAs  is  especially  noteworthy    Collaboration  across  groups  in  the  division  should  make  this  a 
very  valuable  activity  as  the  importance  of  EDFAs  in  deployed  communication  systems  grows. 

The  development  of  wavelength  standards  in  the  1510  to  1540  nm  range  in  the  Fiber  and 
Discrete  Components  project  is  very  important    As  the  effort  is  extended  to  1540  to  1560  nm, 
the  standards  should  prove  invaluable  to  companies  developing  wavelength  division  multiplexing 
components  for  next  generation  fiber  communication  systems    Work  on  fiber  Bragg  grating 
devices  is  of  high  quality  and  interfaces  well  with  the  high-accuracy  wavelength  calibration 
standard. 

In  the  Optical  Fiber  Sensors  project,  work  on  iron  garnet  magneto-optic  sensors  is 
excellent  and  has  been  coordinated  with  a  domestic  materials  supplier    Ongoing  efforts  in  sensor 
standards  and  sensor  components  should  be  examined  for  alignment  with  and  priority  relative  to 
the  division's  plan 

The  Dielectric  Materials  and  Devices  project  has  two  major  focuses.  The  first  is  materials 
metrology    Characterization  tools  being  developed  for  determining  uniformity  of  optical 
properties  on  LiNb03  wafers  are  sorely  needed  by  industry    The  division  is  strongly  encouraged 
to  work  closely  with  industry  in  this  activity    The  other  major  focus,  development  and 
characterization  of  rare-earth  glasses  and  associated  integrated  optic  lasers  and  amplifiers,  is  well 
recognized  and  of  high  quality    This  focus  should  continue  to  have  a  strong  emphasis  on 
waveguide  material  characterization 

The  Semiconductor  Materials  and  Devices  project  focuses  most  of  its  effort  on  epitaxial 
materials  growth  and  characterization  and  has  a  small  effort  in  devices  The  panel  finds  this 
allocation  very  appropriate    The  emphasis  on  in  situ  growth  monitoring  and  spatially  resolved 
surface  and  interface  characterization  of  diode  laser  structures  was  based  on  substantial  industry 
input    Initial  experimental  results  in  these  areas  are  very  encouraging    Close  coupling  with 
industrial  workers  as  this  project  progresses  will  help  ensure  a  high  value  to  US.  industry. 


Assessment  of  Impact  on  Industry 

The  impact  of  the  division's  work  is  reflected  in  publications  such  as  the  recent  scatter 
diagram  showing  results  of  the  multinational  cladding  diameter  measurement  comparison 
{Technical  Digest.  Symposium  on  Optical  Fiber  Measurements  [NIST  SP  864],  1994,  pp   129- 
132,  Fiber  Geometry    Results  of  an  International  Interlaboratory  Measurement  Comparison) 
The  diagram  shows  that  US  manufacturers  are  superior  vendors  because  of  their  tight  process 
control,  and  this  publication  will  give  industry  worldwide  the  incentive  to  converge    Results  of 
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other  programs  have  been  directly  evaluated  by  customer  feedback  from  participating  industries. 
For  example,  wavelength  standards  work  and  the  polarization  standard  were  extremely  well 
received  by  customers. 

In  developing  closer  collaboration  with  industry,  the  division  should  develop  multiple- 
pronged  approaches  separately  tailored  for  both  large  and  small  companies  as  appropriate   This  is 
necessary  given  the  mixed  profile  of  optoelectronics  companies  in  the  United  States 


Recommendations 

The  panel's  fiscal  year  1995  reconmiendations  for  the  Optoelectronics  Division  are  as 
follows 

•  The  impact  of  the  Optoelectronics  Division's  work  should  be  measured  in  terms  of 
improvements  over  time  in  industry  and  by  tracking  the  demand  for  related  services  and 
reference  material  by  the  customers 

•  Since  shifts  in  the  relatively  new  and  dynamic  optoelectronics  industry  are  quite  common, 
these  shifts  should  be  observed  and  followed  as  appropriate   This  action  puts  emphasis  on 
having  and  following  a  somewhat  formal  new  project  selection  process. 

•  The  division  should  develop  multipronged  strategies  to  accommodate  both  large  and  small 
industrial  collaborations 


Optoelectronics  Division  Responses  to  Fiscal  Year  1994  Recommendations 

The  panel's  1994  recommendations  for  the  Optoelectronics  Division  are  given  below, 
quoted  fi"om  the  fiscal  year  1994  assessment,  with  the  division's  response.  The  1995  panel  notes 
that  all  1 994  recommendations  have  been  acted  on.  Most  have  been  implemented,  and  some  are 
in  the  process  of  being  reprioritized  because  of  technology  shifts  in  industry   Most  notably,  the 
division  has  refocused  itself  toward  the  mission  of  metrology  and  away  fi-om  "science  for 
science's  sake." 

1994  Panel  Recommendation:  "The  .  [division]  should  consider  potential  needs  related  to 
displays,  optical  computation/interconnect,  ultrahigh-speed  (tera-bit)  technology,  and  soliton 
measurements"  (p.  41). 

Division  Response:  E>uring  the  latter  part  of  fiscal  year  1994,  the  division  became  more  active  in 
both  intermediate-  and  long-term  program  planning,  and  that  emphasis  continues.  Members  of 
the  division  have  made  many  visits  to  companies  and  universities  to  discuss  advanced  technology 
and  future  requirements.  With  regard  to  the  specific  technologies  noted,  displays  have  not  been  a 
part  of  the  division's  scope,  but  a  part  of  the  Electricity  Division's  Video  Technology  project. 
Optical  computation  and  interconnects  are  of  definite  interest  to  the  division,  and  a  small  effort  in 
data  interconnects  has  begun    Tera-bit  technology  is  being  addressed  in  NIST,  but  mostly  by 
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other  divisions    For  several  years,  the  division  has  been  using  soliton  technology  to  develop  high- 
speed waveform  measurement  systems  based  on  optical  sampling 

1994  Panel  Recommendation:  "The  division  should  support  a  moderate  effort  on  emerging 
technologies  with  fundamental  needs  for  new  measurement  concepts,  including  high-speed 
technology  up  to  60  GHz,  measurement  needs  for  wavelength  division  multiplexing,  and  optical 
storage"  (p.  41). 

Division  Response:  The  division  is  moving  in  the  directions  suggested    The  division  presently 
has  the  capability  to  perform  both  time-  and  frequency-domain  measurement  on  detectors  and 
modulators  to  about  50  GHz  and  is  extending  these  capabilities  to  about  60  GHz    The  new 
division  structure  includes  a  project  on  fiber  and  discrete  devices,  which  is  aimed  in  part  at 
components  for  wavelength  division  multiplexing    The  division  agrees  it  has  an  important  role  to 
play  in  the  optical  data  storage  industry  but  has  not  yet  been  able  to  start  a  program 

1994  Panel  Recommendation:  "The  division  should  maintain  or  expand  significant  efforts  on 
technologies  already  in  the  manufacturing  phase:  (1)  fiber  measurements        (2)  integrated 
optoelectronics  ...  (3)  measurements  for  sources,  detectors,  and  amplifiers  .    .  (4)  power 
measurement  .    .  and  (5)  optical  sensors      ."  (pp  41-42). 

Division  Response:  The  majority  of  the  division's  resources  are  focused  in  these  areas,  and  the 
division  actively  contributes  in  almost  every  area  mentioned    Division  staff  published 
approximately  40  papers  and  gave  approximately  45  talks  during  fiscal  year  1994,  mostly  in  these 
areas    During  the  latter  part  of  fiscal  year  1994  and  continuing  into  fiscal  year  1995,  the  division 
has  been  working  to  understand  better  how  it  currently  allocates  resources  among  these  areas  in 
order  to  provide  a  basis  fi^om  which  it  can  make  adjustments  and  expand  contributions    Part  of 
this  effort  involves  potential  collaborations  with  the  Physics  Laboratory 

1994  Panel  Recommendation:  "The  division  should  continue  to  assess  customer  needs  by 
maintaining  direct  contact  with  all  major  optical  electronics  industries  and  participation  in 
standards  groups"  (p.  42). 

Division  Response:  Division  staff  sit  on  approximately  1 8  domestic  and  international  standards 
committees,  either  as  a  member  or  observer,  and  on  several  professional  society  boards  and 
committees    Staff  members  serve  on  committees  for  several  of  the  most  important  and  relevant 
optoelectronic  conference  committees,  and  staff  attendance  at  conferences  is  high    The  division 
has  about  12  ongoing  collaborations  with  companies    Last  year,  staff  noted  over  75  significant 
telephone  consultations  with  industry    Staff  members  participated  actively  in  the  Optoelectronics 
Industry  Development  Association  roadmap  process    NIST  is  cosponsoring,  and  two  staff 
members  are  participating  in,  a  Japanese  Technology  Evaluation  Center  study  of  the  Japanese  and 
US  optoelectronic  industries,  which  involves  visits  to  about  10  U.S.  and  30  Japanese  companies; 
other  staff  members  regularly  visit  companies  for  informal  exchanges  of  information    All  of  this 
requires  over  1 50  trips  by  staff  each  year,  most  of  which  are  documented  in  trip  reports  and 
available  for  program  planning    Calibrations  services  are  another  source  of  industry  input.  In  a 
typical  year,  the  division  serves  40  to  50  customers  and  discusses  services  with  many  others. 
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1994  Panel  Recommendation:  "The  division  should  develop  an  external  funding  strategy  for 
mature  programs  in  order  to  use  internal  funds  to  expand  support  for  critical  areas,  particularly 
those  entering  the  manufacturing  stage"  (p  42). 

Division  Response:  The  division  definitely  needs  to  develop  a  more  coherent  external  funding 
strategy,  but  it  may  not  be  as  simple  as  using  external  funds  for  more  mature  technologies 
Historically,  the  division  has  been  more  successful  in  identifying  external  funding  for  advanced 
technologies  and,  as  the  technologies  matured,  was  able  to  shift  their  support  to  internal  funding. 
This  was  true  of  work  in  support  of  the  sensor  industry  and  is  anticipated  for  work  in  the 
characterization  of  vertical-cavity  surface-emitting  lasers,  which  recently  received  funding  from 
ARPA    In  other  technologies,  where  the  division  has  obtained  external  funding  for  well- 
established  programs,  contract  work  must  not  divert  efforts  from  advancing  the  state  of  the  art  of 
metrology    The  percentage  of  external  funding  in  this  division  is  not  burdensome  at  present,  and 
the  division  hopes  to  increase  external  funding  in  each  of  the  next  several  years. 

1994  Panel  Recommendation:  "The  division  should  clearly  delineate  the  EEEL  role  relative  to 
that  of  the  Physics  Laboratory  in  detector  characterization  and  narrow-line  laser  diode 
characterization    Industrial  pull  on  the  existing  capability  should  be  a  major  consideration  in 
resolving  issues  of  organizational  overlap"  (p  42) 

Division  Response:  The  respective  roles  of  EEEL  and  the  Physics  Laboratory  are  presently  being 
addressed  at  both  the  laboratory  level  and  division  level    With  respect  to  Optoelectronics 
Division  overlaps  with  the  Radiometric  Physics  Division,  visits  of  staff  have  occurred  in  both 
directions  and  discussions  are  ongoing    It  is  the  position  of  the  Optoelectronics  Division  that 
there  are  three  major  considerations  in  addressing  organization  overlaps:  technical  quality  and 
integrity,  service  to  customers,  and  efficiency 

1994  Panel  Recommendation:  "NIST  requires  a  strategy  for  meeting  the  needs  of  the  laser 
industry  in  industrial  and  medical  applications    The    .    [division]  alone  does  not  have  the 
resources  to  support  needs  ranging  from  short-wavelength  lithography  to  medical  welding  and 
cutting  applications    The  division  role  should  be  defined  in  the  context  of  the  entire  NIST  effort" 
(p.  42). 

Division  Response:  The  Optoelectronics  Division  is  to  be  the  focus  of  optoelectronics  research  at 
NIST    However,  a  dozen  or  so  other  divisions  have  activities  in  optoelectronics  and  facilities  or 
expertise  that  could  be  available  to  support  the  industry    There  is  a  growing  recognition  within 
NIST  that  better  coordination  of  these  activities  could  result  in  greater  support  of  the  industry 
without  additional  resources,  and  therefore  steps  in  this  direction  should  be  a  priority    With 
regard  to  the  ability  of  the  division  to  support  specific  segments  of  the  industry,  the  strategy  is  to 
attempt  to  apportion  division  resources  according  to  industrial  need  and  ability  to  have  an  impact. 


Office  of  Microelectronics  Programs 

The  primary  activity  of  the  Office  of  Microelectronics  Programs  is  the  National 
Semiconductor  Metrology  Program  (NSMP).  The  mission  of  the  NSMP  is  to  apply  NIST 
expertise  to  metrology  needs  of  the  semiconductor  industries,  to  collaborate  with  industry  in 
executing  NIST's  work  and  delivering  its  results,  to  coordinate  metrological  work  among 
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cooperating  government  laboratories,  and  to  provide  industry  a  path  to  government  metrological 
expertise  and  activities  The  funds  made  available  to  this  program  have  grown  from  $15  million 
in  fiscal  year  1991  to  $10  million  in  fiscal  year  1995 

The  panel  applauds  the  strong  collaboration  between  OMP  and  SEMATECH  in  the  SIA 
Roadmap  activities  and  the  critical  ownership  OMP  has  assumed  in  forming  the  NSMP  Funded 
programs  planned  for  1 995  generally  align  with  the  Roadmap,  although  some  programs  succeed 
more  in  this  regard  than  others  The  panel  cautions  that  in  an  environment  of  increasing  budget 
and  new  project  initiation,  there  is  potential  for  programs  to  be  established  without  means  for 
completion  or  termination  based  on  a  global  strategic  plan. 

OMP  priorities  are  clearly  established  based  on  relevance  to  the  SIA  Roadmap  and 
currently  existing  expertise,  examples  include  funded  programs  on  electrical  and  thermal  model 
development  for  integrated  circuits  and  packages    Further  strengthening  of  and  emphasis  on  this 
activity  are  recommended  by  the  panel    The  current  process  for  selection  of  projects  is  to  exploit 
metrology  capabilities  throughout  NIST.  This  initial  approach  is  acceptable,  but  it  is  essential  that 
the  selection  criteria  rapidly  migrate  to  need,  with  current  skills  a  secondary  rather  than  primary 
criteria. 


Office  of  Law  Enforcement  Standards 

The  Office  of  Law  Enforcement  Standards  (OLES)  is  under  new  leadership,  a  change  that 
appears  to  be  a  positive  force  toward  continuing  the  OLES  tradition  of  working  closely  with  law 
enforcement  institutions  and  the  industries  that  serve  them    The  fiscal  year  1994  accomplishments 
as  well  as  work  planned  for  fiscal  year  1995  appear  to  be  closely  linked  to  customer  needs.  The 
need  for  a  modem  firing  range  for  OLES  continues    A  suggestion  is  to  include  such  a  facility  in  a 
new  NIST  building. 
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Mr.  GUTKNECHT.  Thank  you,  Dr.  Rhyne. 
Dr.  Glaze? 

STATEMENT  OF  DR.  JAMES  A.  GLAZE 

Dr.  Glaze.  Thank  you,  Mr.  Chairman  and  Members  of  the  Com- 
mittee. My  name  is  Jim  Glaze.  I  am  the  Vice  President  for  Tech- 
nology Programs  for  the  Semiconductor  Industry  Association.  I  ap- 
preciate the  opportunity  to  be  here  today  to  discuss  the  National 
Technology  Roadmap  for  Semiconductors,  which  my  organization 
has  sponsored,  and  in  particular  the  important  role  that  the  EEEL 
has  played  in  both  the  planning  and  implementation  phases  of  that 
work. 

The  SIA  represents  32  U.S. -based  semiconductor  manufacturers. 
I  think  the  total  manufacturing  production  of  that  group  is  about 
85  or  90  percent  of  all  we  produce  in  this  country.  Some  260,000 
people  work  in  the  semiconductor  manufacturing  area.  Perhaps 
more  importantly,  that  effort  underpins  or  enables  a  $400  billion 
electronics  industry  in  this  country.  That  industry  worldwide,  by 
the  end  of  this  century,  will  be  the  largest  industry  in  the  world. 

Today,  the  U.S.  semiconductor  industry  has  a  lead  in  market 
share  of  about  41  percent  of  the  world  market.  I  believe  Japan  is 
somewhere  around  40,  very  close  to  us.  To  achieve  that  market 
share,  our  industry  has  spent  an  unprecedented  sum  of  money  in 
both  capital  investments  and  research  and  development.  We  spend 
about  20  percent  of  all  revenues  on  capital  and  approximately  11 
percent  on  research  and  development.  That  is  amongst  the  highest, 
if  not  the  highest,  for  any  industry  anywhere. 

A  little  bit  about  the  National  Technology  Roadmap  for  Semi- 
conductors. Basically,  in  reviewing  the  needs  of  the  industry  in 
1992,  the  SIA  Board  decided  to  establish  a  comprehensive  review 
of  all  the  semiconductor  technology  that  would  be  required  into  the 
future  that  would  provide  a  basis  for  future  growth  of  the  industry. 
As  you  know,  semiconductor  manufacturing  is  almost  unimaginably 
technology  intensive. 

It  was  clear  at  that  point,  when  that  early  1992  initiative  was 
established,  that  individual  companies  working  individually  would 
not  be  able  to  develop  the  methodologies  and  develop  the  pathways 
necessary  for  our  industry  to  move  forward,  just  because  of  the 
vast  technology  content. 

In  November  of  1992,  basically,  technical  experts  from  all  parts 
of  the  semiconductor  industry,  including  the  producers,  the  suppli- 
ers, the  customers,  the  government,  and  universities,  got  together 
to  develop  what  was  called  a  common  vision  of  the  course  of  semi- 
conductor technology  over  the  next  15  years.  This  product  was  basi- 
cally called  the  National  Technology  Roadmap  for  Semiconductors. 

In  1994,  SIA  reconvened  this  group,  some  300  or  more  people, 
and  revised  this  Roadmap  to  extend  it  one  more  generation  beyond 
the  1992  time  line.  Basically,  from  a  technical  point,  that  extends 
the  evolution  of  semiconductors  down  to  0.07  micron  sizes,  an  al- 
most unheard  of  number,  from  today's  0.35  micron  design  rule. 

The  purpose  of  the  Roadmap  is  to  provide  a  quality  database  of 
needs  and  a  framework  by  which  the  semiconductor  industry  and 
those  in  government  and  academia  associated  with  the  semiconduc- 
tor technology  can  systematically  approach  this  enormous  research 
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and  development  requirement.  SIA's  intent  is  for  the  Roadmap  to 
address  a  common  national  need  for  achieving  both  commercial 
competitiveness  and  defense  preparedness,  dual  use,  if  you  will. 

There  are  many  internal  programs  in  microelectronics  sponsored 
by  the  Departments  of  Defense,  Commerce,  and  Energy.  The  De- 
partment of  Commerce,  through  its  NIST  laboratory,  has  played  a 
key  role  in  developing  standards  in  other  cross-cutting  technologies 
that  are  of  key  importance  to  our  industry.  Not  only  that,  they 
have  played  a  key  role  in  defining  these  requirements  and  have 
played  a  key  role  in  managing  the  roadmap  process.  I  believe  the 
Director  of  EEEL  sits  on  our  Roadmap  coordinating  group. 

This  agency,  it  is  our  opinion,  should  continue  to  be  the  focal 
point  for  development  of  advanced  standards  and  metrology  tech- 
nologies and  should  represent  a  stake  in  the  ground  for  our  indus- 
try on  these  areas. 

There  are  three  aspects  of  NIST  that  are  critical  to  our  industry. 
Number  one,  NIST's  technology  standards  formulation,  implemen- 
tation, and  dissemination  capabilities,  which  have  been  discussed 
at  length  before. 

Two,  NIST's  ability  to  work  with  foreign  governments.  Tom 
Rhjme  has  expressed,  standards  are  often  used  by  countries  as  a 
non-tariff  barrier. 

Finally,  NIST's  scientists.  NIST's  staff  is  a  valuable  expert  re- 
source for  our  industry.  The  development,  maintenance,  and  dis- 
semination of  standards  is  a  complex  and  difficult  task  which  is  far 
too  large  for  any  single  company  to  undertake.  If  such  a  company 
were  to  undertake  it,  it  would  no  longer  be  a  pre-competitive  but 
it  would  become  a  competitive  force,  which  would  be  for  the  dis- 
advantage of  our  entire  industry.  NIST's  staff  acts  as  expert,  inde- 
pendent arbitrators  to  resolve  difficult  programs  in  setting  or  using 
these  important  standards. 

The  current  National  Semiconductor  Metrology  Program  under 
EEEL  was  formally  estabUshed  by  NIST  in  1994.  It  had  its  roots 
in  a  semiconductor  initiative  begun  earlier,  in  1991.  Cognizant  of 
the  fact  that  the  production  of  semiconductors  require  numerous 
measurements  and  the  semiconductor  devices  are  becoming  in- 
creasingly complex  so  that  the  industry's  measurement  needs  are 
constantly  becoming  more  complex,  as  well. 

If  you  look  at  the  way  we  break  out  the  various  tasks  within  the 
National  Roadmap,  we  break  them  out  into  eight  technology  focus 
areas.  Every  single  one  of  those  areas  have  substantial  require- 
ments for  the  metrology  and  standards,  and  so  the  participation  by 
NIST  has  cut  across — that's  why  we  call  it  cross-cutting — every  sin- 
gle area  of  our  thinking  and  planning  on  the  technology  pathways 
to  future  progress. 
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I  would  say,  to  this  end,  the  EEEL  and  the  National  Semiconduc- 
tor Metrology  Program  serve  a  very  critical  function  for  our  indus- 
try and  we  strongly  support  continued,  or,  I  should  say,  future  com- 
mitment to  the  organization  and  to  the  National  Semiconductor 
Metrology  Program.  We  would  like  to  see  in  1997  full  funding,  as 
recommended  by  the  SIA  and  others,  of  at  least  $25  million  a  year. 
This  is  a  relatively  small  number,  but  the  leverage  that  we  gain 
from  this  organization  gives  many,  many  times  the  amount  of 
money  this  country  invests.  Thank  you. 

[The  prepared  statement  of  Dr.  Glaze  follows:] 
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STATEMENT  OF 

JAMES  A.  GLAZE 
VICE  PRESIDENT,  TECHNOLOGY  PROGRAMS, 
SEMICONDUCTOR  INDUSTRY  ASSOCIATION 

JUNE  25,  1996 


Madam  Chairwoman,  my  name  is  James  A.  Glaze,  and  I  am  the  Vice 
President  for  Technology  Programs  at  the  Semiconductor  Industry  Association 
(SIA).  I  appreciate  the  opportunity  to  share  with  the  Subcommittee  SIA's  views 
on  the  evolution  and  utility  of  the  National  Technology  Roadmap  for 
Semiconductors,  the  participation  of  the  Electronics  and  Electrical  Engineering 
Laboratory  (EEEL)  of  the  National  Institute  of  Standards  and  Technology  (NIST) 
in  the  Roadmap  process,  and  the  role  of  EEEL  in  meeting  the  metrology  needs 
of  the  U.S.  semiconductor  industry. 

SIA  represents  32  U.S. -based  semiconductor  manufacturers.  Since  its 
founding,  SIA  has  been  actively  engaged  in  issues  of  technology  development 
and  policy  as  they  affect  the  industry  as  a  whole. 


The  U.S.  Semiconductor  Industry 

Semiconductors  are  an  increasingly  pervasive  aspect  of  everyday  life, 
enabling  the  creation  of  the  information  superhighway  and  the  functioning  of 
everything  from  automobiles  to  advanced  medical  equipment.  Semiconductors 
are  also  the  linchpin  underlying  this  nation's  advanced  military  weapons 
systems.  A  growing  proportion  of  the  value  of  these  systems  is  dependent 
upon  electronics  products  -  up  to  40  percent  in  some  cases.  The  current 
design  of  the  F-16  Fighter,  for  example,  includes  17,000  electronics 
components.  They  are  also  intrinsically  important  in  radars,  weapons  guidance 
and  control  systems. 

U.S.  semiconductor  makers  employ  260,000  people  nationwide.  Their 
products  are  the  enabling  technology  behind  the  nearly  $400  billion  U.S. 
electronics  industry,  which  provides  employment  for  2.5  million  Americans. 

The  U.S.  semiconductor  industry  is  currently  the  world  market  share 
leader,  with  1995  world  sales  reaching  $59  billion,  representing  almost  41 
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percent  of  the  $144  billion  world  market.  Moreover,  the  world  semiconductor 
market  is  expected  to  double  by  the  year  2000,  with  projected  sales  of  over 
$300  billion. 

U.S.  semiconductor  producers  are  highly  committed  to  maintaining  their 
lead  in  both  semiconductor  manufacturing  and  technology.  The  U.S. 
semiconductor  industry  devotes  on  average  20  percent  of  its  revenues  to 
capital  spending  and  another  1 1  percent  to  research  and  development  -among 
the  highest  of  any  U.S.  industry. 


The  National  Technology  Roadmap  for  Semiconductors 

Background:  With  the  formation  of  SEMATECH  in  1987,  SIA  greatly 
expanded  its  efforts  in  supporting  technology  development  for  the  industry. 
Combined  with  the  Semiconductor  Research  Corporation  (SRC),  an  SIA 
consortium  that  supports  university  research,  the  resulting  impact  on  improving 
the  technical  capabilities  in  the  industry  was  substantial.  In  reviewing  the 
needs  of  the  industry  in  1992,  the  SIA  Board  decided  to  establish  a 
comprehensive  review  of  semiconductor  technology  and  provide  a  basis  from 
which  industry,  government,  and  academia  could  cooperate.  This  built  upon 
the  work  of  the  National  Advisory  Committee  on  Semiconductors  (NACS). 

In  November  of  1992,  technical  experts  from  all  parts  of  the 
semiconductor  industry,  including  semiconductor  producers,  suppliers  and 
customers,  government,  and  universities  came  together  to  develop  a  common 
vision  of  the  course  of  semiconductor  technology  over  the  next  fifteen  years. 
The  product  from  this  pioneering  effort  was  the  National  Technology  Roadmap 
for  Semiconductors  detailing  industry  trends,  requirements,  and  obstacles  for 
broad  segments  of  semiconductor  manufacturing  technology. 

Acceptance  and  utilization  of  the  technology  Roadmap  by  the 
semiconductor  community  has  been  overwhelming.  In  1994,  the  Roadmap 
was  revised  to  extend  the  timeline  one  more  generation  beyond  that  of  the 
1992  Roadmap  to  .07  micron  technology  requirements  in  2010. 

Policy:  The  purpose  of  the  Roadmap  is  to  provide  a  quality  database  of 
"needs"  and  a  framework  by  which  the  semiconductor  industry  and  those  in 
government  and  academia  associated  with  semiconductor  technology  can 
systematically  approach  the  enormous  research  and  development  required  to 
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meet  those  needs.  "Needs"  are  requirements  on  technology  that  allow  the 
semiconductor  manufacturing  and  supplier  industries  to  maintain  performance 
growth.  The  Roadmap  defines  these  needs  hierarchically.  First,  overall 
roadmap  technology  characteristics  are  set  forth  to  show  the  top  level  of  needs. 
Then  individual  technology  roadmap  sections  prepared  by  the  eight 
Technology  Working  Croups  (TWGs)  define  additional  levels  of  needs  specific 
to  each  technology  area. 

Each  TWG  represents  one  of  the  technologies  of  direct  interest  to 
semiconductor  product  development  and  manufacturing  operations.  These 
critical  technologies  are:  (1)  Design  and  Test,  (2)  Process  Integration,  Devices, 
and  Structures,  (3)  Environment,  Safety,  and  Health,  (4)  Lithography,  (5) 
Interconnect,  (6)  Materials  and  Bulk  Processes,  (7)  Assembly  and  Packaging, 
and  (8)  Factory  Integration. 

The  TWGs  have  also  identified  technology  needs  that  extend  into  related 
disciplines  closely  aligned  with  the  interests  of  suppliers,  universities,  and 
national  laboratories.  These  "Crosscut  Technologies"  include  metrology, 
modeling  and  simulation,  electronic  materials,  standards,  contamination-free 
manufacturing,  and  quality  and  reliability. 

Because  it  is  important  to  know  what  technology  approaches  are 
currently  under  consideration,  in  addition  to  the  needs  identified  by  the  TWGs, 
the  Roadmap  also  identifies  some  potential  or  alternative  technologies. 
However,  it  should  be  emphasized  that  the  Roadmap  does  not  dictate  which 
options  are  preferred  or  which  should  be  pursued  by  the  research  community 
(and  their  funding  agencies).  Instead,  the  intent  of  the  Roadmap  is  to  enhance 
communication  and  facilitate  innovation  to  meet  the  needs  identified  in  the 
Roadmap. 

SIA's  intent  for  the  Roadmap  is  to  address  a  common  national  need  of 
achieving  both  commercial  competitiveness  and  defense  preparedness  (dual- 
use).  Moreover,  this  national  need  can  best  be  achieved  through  the 
cooperative  efforts  of  participating  technology  leaders  from  industry, 
government,  and  academia.  This  can  provide  a  solid  foundation  for 
independent  or  collaborative  efforts  throughout  the  nation  to  advance 
semiconductor  technology.  Only  through  government  participation  as  a 
partner  in  this  technology  assessment  process  can  the  semiconductor 
technology  base  be  fully  and  effectively  exploited. 
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This  policy  also  challenges  the  industry  to  achieve  its  technology 
objectives  through  existing  organizations  and  with  no  significant  increases  in 
the  current  levels  of  investment  as  a  percentage  of  total  output.  It  is  SIA's  goal 
to  make  the  best  use  of  available  U.S.  resources,  encourage  the  alignment  of 
government  programs  consistent  with  the  Roadmap,  and  rely  upon  the  existing 
national  research  structures  and  institutions  to  implement  recommendations. 

Of  course,  in  a  field  as  dynamic  as  semiconductor  technology,  it  is 
impossible  to  predict  the  future.  To  be  of  ongoing  utility,  the  Roadmap  will 
require  regular  updating  as  the  future  terrain  becomes  more  clearly  visible. 
SIA's  plan  is  to  examine  the  Roadmap  regularly  -  probably  by  reconvening 
technical  experts  every  couple  of  years  to  keep  it  current  and  vital.  The  goal 
is  to  have  a  set  of  charts  that  all  parties  working  within  or  associated  with  the 
industry  can  consult  when  examining  their  research  and  development 
priorities. 

Strategy:  The  Roadmap  provides  a  framework  that  can  enhance 
collaboration,  increase  shared  knowledge,  and  minimize  duplication  of  efforts. 
But  implementation  of  the  Roadmap  will  require  investments  by  both  industry 
and  government.  Investment  vehicles  will  include  internal  research  and 
development  programs  by  semiconductor  manufacturers  and  their  suppliers, 
industry  and  industry/government  consortia,  government  agencies  and  special 
government/industry  programs  that  target  critical  civilian,  defense,  and  defense 
conversion  needs.  Below  is  a  summary  of  some  of  the  programs  that  address 
Roadmap  objectives. 

Industrial  R&D:  The  U.S.  semiconductor  industry  and  its  suppliers  invest 
around  1 1  percent  of  their  annual  revenues  on  internal  R&D.  This  rate  of 
expenditure  is  among  the  highest  of  any  industry  in  the  nation.  For  the  most 
part,  these  internal  programs  address  technologies  that  can  be  incorporated  into 
manufacturing  over  a  one  to  three  year  time  frame.  This  industry  invests  more 
than  $5  billion  dollars  annually  in  R&D,  including  over  $90  million  annually 
in  SEMATECH  and  over  $35  million  in  SRC. 

Government  Agencies:  Internal  programs  in  microelectronics  are 
sponsored  by  the  Departments  of  Defense,  Commerce,  and  Energy.  The 
Department  of  Defense,  especially  through  ARPA,  has  played  a  vital  role  in  the 
development  of  advanced  microelectronic  technologies.  These  programs  - 
ones  that  have  a  five  to  ten  year  time  horizon  -  best  serve  the  Roadmap 
mission  by  addressing  precompetitive  high-risk  technologies  with  no  short-term 
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commercial  return  and  that  are,  therefore,  often  beyond  the  means  of  industry 
alone  to  support.  The  Department  of  Commerce,  through  its  NIST  laboratory, 
has  played  a  key  role  in  developing  standards  and  other  "cross-cutting" 
technologies  in  support  of  microelectronics  manufacturing.  This  agency  should 
continue  to  be  the  focal  point  for  development  of  advanced  standards  and 
metrology  technologies.  The  Department  of  Energy,  through  the  National 
Laboratories,  has  sponsored  numerous  electronics  programs  in  support  of  their 
defense  mission.  These  programs  are  now  becoming  available  to  the 
commercial  sector.  Moreover,  laboratory  leaders  are  actively  aligning  these 
programs  to  the  Roadmap. 


The  National  Institute  of  Standards  and  Technology 

Because  of  the  fundamental  contributions  technology  is  making  to  the 
nation's  security,  culture  and  economic  growth,  SIA  believes  it  is  essential  that 
the  U.S.  government  have  an  agency  with  the  technical  depth  to  understand 
and  monitor  advanced  technologies.  This  function  is  currently  performed  by 
the  Technology  Administration  of  the  Department  of  Commerce. 

The  agency  within  the  Technology  Administration  whose  work  is 
essential  to  the  SIA  and  the  nation  is  the  National  Institute  of  Standards  and 
Technology  (NIST).  Created  as  the  National  Bureau  of  Standards  in  1901, 
NIST's  mandate  is  "to  help  U.S.  industry  improve  its  competitiveness  through 
new  technologies,  modernized  production  presence,  improved  quality  control, 
and  rapid  commercialization." 

Three  aspects  of  NIST  are  critical  to  the  U.S.  semiconductor  industry: 

1 .  NIST's  technology  standards  formulation,  implementation  and 
dissemination  capabilities.  U.S.  industry  must  have  nationally  and 
internationally  recognized  technology  standards  for  its  products.  As  technology 
gets  more  complex  and  diverse,  the  need  for  accurate  standards  is  growing. 
For  example,  the  semiconductor  industry  achieves  its  increasing  productivity 
in  part  by  printing  progressively  smaller  features.  When  in  about  five  years  the 
printed  feature  size  of  semiconductors  reaches  about  0.18  micron,  industry  will 
not  have  a  standard  method  of  accurately  measuring  these  features  in 
production.  As  discussed  further  below,  joint  NIST/industry  program  is 
addressing  this  problem. 
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2.  NIST's  ability  to  work  with  foreign  governments.  Standards  are  often 
used  by  countries  as  nontariff  trade  barriers.  Often,  the  key  to  having  a  new 
system  accepted  in  the  world  marketplace  is  establishing  a  worldwide  standard. 
Without  such  a  standard,  a  market  is  fragmented  and  market  development  is 
impeded.  Countries  frequently  require  imported  foreign  instrumentation  and 
components  to  be  re-certified,  resulting  in  extra  delay  and  cost.  A  country  that 
can  develop  a  standard  with  its  industry  acceptable  worldwide  can  avoid  the 
import  re-certification  requirement  and  gain  a  significant  market  advantage. 
NIST  participates  in  government-to-government  standard  setting  activities  and 
thereby  assists  U.S.  industry  and  technology  development  generally  by 
ensuring  that  new  standards  meet  U.S.  interests. 

3.  NIST's  scientists.  NIST's  staff  is  a  valuable  expert  resource  for  U.S. 
industry.  The  development,  maintenance  and  dissemination  of  standards  is  a 
complex  and  difficult  task  which  is  too  large  for  a  single  company,  and  often 
is  resisted  by  others  in  the  industry  because  it  gives  the  controlling  company 
a  competitive  advantage.  NIST's  staff  act  as  expert,  independent  arbitrators  to 
resolve  difficulties  in  setting  or  using  standards. 


The  National  Semiconductor  Metrology  Program 

The  current  National  Semiconductor  Metrology  Program  (NSMP)  - 
formally  established  by  NIST  in  1994  -  had  its  roots  in  a  semiconductor 
initiative  begun  in  1991.  Funding  for  the  NSMP  doubled  in  FY  1995  to  $10 
million.  While  SIA  supported  a  budget  of  $19  million  for  FY  1996,  no  funds 
were  appropriated  for  this  fiscal  year. 

The  production  of  semiconductors  requires  numerous  measurements,  and 
as  semiconductor  devices  are  becoming  increasingly  complex,  so  the  industry's 
measurement  needs  are  constantly  becoming  more  complex  as  well.  The 
technology  challenges  facing  the  industry  as  semiconductor  chips  get  smaller 
and  smaller  require  measurement  and  characterization  capabilities  well  beyond 
those  now  available.  To  this  end,  the  NSMP  serves  a  critical  function  for  the 
industry,  as  well  as  for  its  customers  both  in  the  private  sector  and  the 
government  -  especially  the  Department  of  Defense.  SIA  therefore  supports 
funding  for  the  program  of  FY  1 997  of  $25  million.  As  the  funding  policies  of 
both  ARPA  and  SEMATECH  have  recently  changed,  even  at  this  full  funding 
level,  the  program's  costs  are  small  in  comparison  with  both  the  size  and  needs 
of  semiconductor  technology  development  in  the  United  States. 
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The  NSMP  supports  measurement  development  programs  in  a  number 
of  important  areas  for  the  semiconductor  industry,  including: 

•  trace  moisture  in  process  gases; 

•  pressure,  composition,  and  flow  rate  of  process  gases; 

•  physical   and   chemical   properties   data  for  modeling  chemical 
reactions; 

•  boron  implants  in  silicon; 

•  surface  microroughness  of  silicon  wafers; 

•  intensity   of  ultraviolet   light  for   making  fine   patterns   on   water 
surfaces; 

•  dimensional  measurements  for  patterns  on  the  surface  of  silicon 
wafers; 

•  properties  of  polymers  for  device  packages;  and 

•  improved  high-temperature  thermocouples. 

As  a  result  of  the  NSMP,  a  number  of  advances  in  metrology  of  value  to 
the  semiconductor  industry  have  been  made,  including:  the  establishment  of 
highly  accurate  electrical  measurements  of  alignment  between  successive  layers 
of  materials,  equipment  to  determine  absolute  moisture  measurements  in  gases 
to  extremely  low  levels,  standard  reference  materials  for  oxygen  and  boron  in 
silicon,  calibration  services  for  ultraviolet  lightmeters,  greater  knowledge  of 
electrically  driven  film  deposition  processes,  and  a  better  understanding  of  the 
manner  in  which  scattered  light  from  highly  polished  surfaces  inhibits  the 
detection  of  small  contaminating  particles. 

The  Electronics  and  Electrical  Engineering  Laboratory  at  NIST 

The  Electronics  and  Electrical  Engineering  Laboratory  (EEEL)  at  NIST 
provides    metrology   support   for  the  electronics    industry   in   areas   such   as 
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electronic    instrumentation,     semiconductors,     magnetics,    superconductors, 
microwaves,  lightwaves  and  video  display  technology. 

EEEL  has  provided  assistance  in  the  development  of  measurement 
technology  to  the  semiconductor  industry  for  forty  years.  For  example,  the 
EEEL  microwave  program  developed  new  measurement  techniques  based  on 
transmission  line  test  structures  formed  directly  on  semiconductor  wafers.  This 
work  uncovered  significant  errors  in  traditional  measurements.  The  U.S. 
industry,  through  adoption  of  these  new  measurement  techniques,  has  greatly 
increased  its  productivity. 

Most  importantly  to  our  industry,  EEEL  is  leading  the  NIST  interlaboratory 
effort  to  develop  metrology  to  meet  the  needs  of  the  semiconductor  industry 
as  identified  in  the  Roadmap.  The  Office  of  Microelectronics  Programs  (OMP) 
at  EEEL  serves  as  the  manager  of  the  National  Semiconductor  Metrology 
Program.  OMP  is  responsible  for  maintaining  the  relevance  of  the  program's 
specific  projects  through  direct  industry  contacts  and  regular  interaction  with 
industry  consortia  such  as  SEMATECH  and  SRC.  OMP  is  also  active  in  the 
Roadmap  process  through  its  participation  in  the  Metrology  Crosscut  Working 
Group  of  the  SIA.  The  Director  of  EEEL  also  serves  as  a  member  of  the 
Roadmap  Coordination  Group.  Each  year  OMP  also  participates  in  SEMICON 
West  -  the  largest  semiconductor  industry  trade  show. 

Currently,  there  are  23  projects  being  undertaken  through  the  NSMP. 
These  projects  are  all  selected  in  conjunction  with  the  semiconductor 
technology  Roadmap  and  specific  industry  input  to  the  various  NIST 
laboratories. 

Significant  accomplishments  of  the  program  include: 

•  Demonstration  of  a  high  resolution  x-ray  spectrometer  which  will 
allow  superior  chemical  identification  of  small  particles  and  light 
elements  while  preserving  the  convenient  operation  necessary  for 
practical  use  in  semiconductor  fabrication. 

•  Development  of  the  first  practical  tool  for  measuring  flatness  on 
300mm  wafers  by  adapting  a  technique  well-known  in  the  optics 
industry  to  silicon. 
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•  Development  of  an  improved  temperature  measurement  device 
(thermocouple)  which  is  much  more  stable  than  existing  devices 
while  also  providing  three  times  the  accuracy  of  current 
thermocouples  at  high  temperatures  -  a  critical  ability  in  the 
semiconductor  industry. 

•  Development  of  new  measurement  methods  designed  to  prevent 
unpredictable  wire  bond  failures  -  a  major  system  reliability 
problem  for  the  industry. 

A  critical  area  requiring  the  continued  focus  of  the  NSMP  is  in  the  field 
of  metrology  for  next  generation  optical  lithography.  As  outlined  in  the 
Roadmap,  the  industry  is  in  need  of  suppliers  that  will  be  able  to  produce  a 
viable  advanced  lithography  tool  to  meet  the  industry's  expected  need  for  180 
to  130  nanometer  lithographic  technology.  EEEL  is  working  closely  with  the 
semiconductor  industry  and  our  suppliers  to  see  that  this  need  can  be  met. 

As  the  above  examples  demonstrate,  the  National  Semiconductor 
Metrology  Program  is  serving  important  needs  of  the  semiconductor  industry 
which  are  not  being  met  elsewhere.  This  program  is  critical  for  the  future 
competitiveness  of  the  U.S.  semiconductor  industry.  It  should  be  fully  funded 
at  $25  million  for  FY  1997. 


Conclusion 

Technology  is  the  driver  for  economic  growth  in  the  world  market  to  an 
unprecedented  degree.  The  Asians  and  Europeans  have  the  same  "short  list" 
of  critical  technologies  as  we  do,  including  semiconductors. 

The  semiconductor  Roadmap  provides  the  medium  to  focus  efforts 
throughout  the  nation  on  semiconductor  technology  development.  But  it  is 
essential  that  industry  and  government  work  together  as  partners  so  that  the 
Roadmap  can  be  implemented  in  a  way  that  benefits  our  nation  from  both  an 
economic  and  security  standpoint. 
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Mr.  GUTKNECHT.  Thank  you,  Dr.  Glaze. 

I  would  now  turn  to  my  colleague,  Mr.  Tanner.  Do  you  have  some 
questions? 

Mr.  Tanner.  Thank  you,  Mr.  Chairman. 

I  just  wish  the  rest  of  the  Congress  could  have  heard  this  panel's 
testimony.  I  want  to  thank  you  for  being  here.  I  understand.  Dr. 
Hebner,  you  came  back  from  Europe.  I  got  back  from  Sarajevo  last 
night  myself,  so  it's  about  8:30,  9:00  to  me,  too. 

Mr.  Hebner.  I  understand  how  you  feel. 

Mr.  Tanner.  Dr.  Rhyne,  thank  you  for  your  testimony.  As  I  said 
in  my  opening  statement,  NIST  is  sort  of  the  stealth  agency  of  the 
Federal  Government.  Not  that  many  people  come  into  contact  with 
it  day  to  day,  and  yet  it  probably  characterizes  more  than  any 
other  agency  that  comes  to  mind  the  idea  that  government  ought 
to  be  an  ally  of  business  rather  than  an  adversary  and  ought  to 
help  American  businesses  compete  worldwide  as  the  government, 
and  the  government  can  do  something  that  is  useful  and  construc- 
tive and  contributes  to  the  quality  of  the  life  of  our  citizens. 

I  think  your  testimony,  better  than  I  can  say,  points  out  just  how 
critical  the  role  that  NIST  plays  as  it  relates  to  our  standing  in  the 
world.  Some  way  or  another,  we  are  going  to  try  to  continue  to  get 
that  word  across  to  people  who  have  a  different  view. 

Let  me  ask  this  question.  The  Omnibus  Trade  and  Competitive- 
ness Act  of  1988  expanded  the  NIST  mission  a  little  bit  beyond  the 
traditional  standards  £ind  metrology  emphasis  to  more  directly  fa- 
cilitate improvements  in  the  competitiveness  of  U.S.  industry.  I 
know  that  we  are  talking  about,  with  this  panel,  the  laboratories 
and  what  the  laboratories  have  done,  and  I  certainly  support  every- 
thing you  all  have  said  and  will  do  what  I  can  to  support  the  fund- 
ing for  this  very  vital  work  that  is  done. 

But  I'm  also  interested  in  some  of  the  programs,  the  outreach 
programs  at  NIST  to  facilitate  worldwide  economic  competitiveness 
by  American  industry.  I  guess  my  question  is,  in  your  opinion,  has 
this  activity  diminished  the  ability  of  NIST  to  deliver  these  more 
traditional  services?  Have  I  characterized  my  inquiry  good  enough? 
I  guess  to  Dr.  Rhyne 

Dr.  Rhyne.  We  agreed  that  Bob  was  going  to  answer  all  the 
questions. 

[Laughter.] 

Mr.  Tanner.  What  I'm  trying  to  get  at 

Dr.  Rhyne.  I  understand.  I  don't  deal  often  in  the  names  of  the 
government  acts,  but  I  assume  you're  talking  about  programs  like 
the  MEP  and  the  ATP  and  the  TRP.  When  I  was  at  MCC,  I  was 
involved  in  about  14  of  the  original  proposals  that  came  out  of  that 
group  as  a  proposal  manager  and  I  think  that  those  provided  an 
excellent  way  for  us  at  MCC  to  bring  together  kind  of,  at  a  nuclea- 
tion  site,  a  set  of  companies  who  otherwise  would  have  either  not 
dealt  with  that  issue  or  they  would  have  been  competitors  with  less 
resources  to  be  available. 

So  I'm  familiar  with  the  early  days  of  what  that  program  did 
within  industry  and  I'm  very  familiar  with  the  role  that  NIST 
played  both  in  administrating  it  and  facilitating  it.  As  best  I  under- 
stand the  organizational  structure  within  NIST,  there's  enough — 
there's  probably  the  appropriate  amount  of  overlap  and  the  appro- 


148 

priate  amount  of  separation  between  individuals  who  are  involved 
in  managing  and  administering  those  programs. 

I  can  say  clearly  that  with  regard  to  EEEL,  that  we  had  the 
same  concern  I  believe  you  voiced.  Would  these  big,  high-funded, 
high  visibility  programs  detract  from  the  basic  metrology  and 
standardization  efforts?  I  don't  think  we've  seen  such  a  detraction 
up  until  the  1995  assessment.  I'm  not  hinting  that  there  was  one. 
As  you  know,  because  of  some  circumstances  here,  we  have  had 
only  a  very  informal  assessment  process  thus  far  in  1996. 

In  fact,  I  think  the  way  in  which  a  small  percentage  of  those 
monies  actually  flowed  out  of  those  programs  to  be  made  available 
for  sort  of  seed  projects  and  other  commercialization  projects  within 
the  EEEL  have  really  been,  to  some  degree,  a  benefit.  Sure,  they 
have  siphoned  off  some  very  good  people  to  go  over  and  administer 
it,  but  in  the  main,  I  think  that  NIST  has  done  a  very  good  job  of— 
I  think  of  the  little  python  swallowing  the  big  pig.  It's  been  a  very 
big  effort  to  administer,  but  I  don't  believe  to  tins  point  that  I  could 
report  to  you  any  diminution  of  the  quality  of  the  work  on  the  basic 
NIST  mission  within  the  EEEL. 

Mr.  Tanner.  Thank  you. 

Dr.  Glaze,  do  you  have  a  comment  on  that? 

Dr.  Glaze.  I  don't  have  any  insight  into  the  inner  workings.  I 
look  at  it  from  the  outside.  Our  interaction  has  primarily  been 
through  the  metrology  and  standards  setting  activities  of  the  Na- 
tional Metrology  Center.  That  effort  has  been  magnificent.  It's  been 
slowed  by  slowing  of  funding,  but  I  don't  see  any  lack  of  enthu- 
siasm or  any  lack  of  progress  from  our  point  of  view  as  a  result  of 
any  of  these  other  programs. 

I'm  aware  of  the  other  programs  and  generally  know  how  they 
work,  but  I  haven't — in  fact,  until  you  asked  the  question,  it  never 
occurred  to  me  that  there  would  have  been  a 

Mr.  Tanner.  I  don't  think  there  is,  either,  but  there  are  some 
who  have  a  different  view  of  that  than  you  and  I. 

Let  me  ask  this.  I  agree  with  you  that  the  Roadmap  for  semi- 
conductors has  been  constructive.  In  your  opinion,  would  this  serve 
as  a  model  for  maybe  some  other  high  technology  fields?  Is  there 
a  place  that  we  could  go  to  work  and  maybe  help  NIST  in  other 
areas  that  we  want  the  United  States  industrial  base  to  be  a  world 
leader  in? 

Dr.  Glaze.  Yes.  It  already  has.  I  think  the  NEMI  activity,  the 
National  Electronics  Manufacturing  Initiative,  which  is  sort  of  a,  if 
you  were  to  imagine  the  Semiconductor  Roadmap  activity,  basi- 
cally, that  model  was  used  to  extend  across  a  vast  area  of  elec- 
tronics, from  microwaves  to  batteries  to  all  aspects  of  electronics. 
They  basically  were  following  the  format.  As  far  as  I  understand, 
NIST  is  playing  a  very  similar  role  with  regard  to  this  initiative. 

Also,  if  you  take  a  piece  of  that,  in  the  electronic  packaging  area, 
it  turns  out  that  electronic  packaging,  while  it  sounds  mundane,  is 
becoming  a  very,  very  big  product  enabler  and  value-added  compo- 
nent to  our  industry.  We  lag  the  rest  of  the  world  in  this  tech- 
nology area.  Recently,  there  has  been  a  major  packaging  effort  that 
has  used  the  Roadmap  formula  and  has  brought  together  all  of  the 
packaging  houses  in  the  United  States  into  another  Roadmapping 
activity,  very  similar  to  this  semiconductor  activity,  and  I  believe 
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NIST  will  be  playing  a  role,  if  they  are  not  already,  in  a  very  simi- 
lar way. 

Fundamentally,  when  these  Roadmapping  activities  get  together 
and  they  turn  to  the  metrology  and  to  the  standards  area,  there's 
no  place  else  to  turn  because  no  single  company  has  that  mandate. 
So  I  wouldn't  know  where  to  go,  setting  up  this  Roadmap. 

Mr.  Tanner.  Yes,  sir? 

Dr.  Rhyne.  I'll  quote  from  the  1995  assessment  of  the  EEEL, 
where  the  panel,  as  a  broad  recommendation  about  the  overall  lab- 
oratory, stated,  "EEEL  should  expand  its  activities  to  assist  addi- 
tional industrial  groups  in  the  electronics  and  electrical  engineer- 
ing sectors  in  defining  their  need  for  metrology  aind  related  serv- 
ices, parallelling  the  work  of  the  SIA  and  the  SI  A  Roadmap." 

In  some  sense,  the  strength  of  the  SIA  program  can  be  viewed 
as  a  weakness  in  that  they  are  so  effective  at  extracting  from  the 
EEEL  the  services  and  support  that  they  need  that  it  has  been  a 
recurring  concern  of  the  panel  that  less  organized  areas  of  the  U.S. 
industry  aren't  able  to  get  that  same  level  of  support.  I  can't  cite 
but  a  couple  of  examples. 

I  think  in  electromagnetic  interference,  in  one  of  the  video  stand- 
ards areas  and  in  a  couple  of  other  areas,  I  think  at  least  with  the 
urging  of  the  panel,  the  NIST  EEEL  management  has  begun  to  try 
to  provide  workshops  which  will  bring  together  leaders  in  those  in- 
dustries to  see  if  they  can't  use  the  model  that's  been  put  forward 
and  used  so  effectively  by  the  SIA  to  help  define  what  their  needs 
should  be  and  what  they  should  be  getting  from  EEEL.  So  I  think 
I  totally  agree  with  you. 

Mr.  Tanner.  Dr.  Hebner,  do  you  have  anything  to  add? 

Mr.  Hebner.  Two  minor  points.  One,  I  guess  I  should  mention 
that  Jud  French,  the  Director  of  the  Electronics  and  Electrical  En- 
gineering Laboratory,  is  unable  to  be  with  us  today  because  he  is 
at  the  NEMI  meeting  to  help  them  understand  what  the  metrology 
needs  are  in  the  electronics  manufacturing. 

Second,  I  should  also  emphasize  that  we  very  carefully  view  our 
role  in  this  area  as  working  with  industry.  What  we  are  trying  to 
do  is  to  encourage  industry  to  think  clearly  about  where  they  need 
to  go  and  what  technological  advances  they  intend  to  make  and 
then  our  role  is  to  look  at  what  they  intend  to  do  technically  and 
try  to  understand  and  extract  what  that  means  for  our  national 
measurement  system  and  to  provide  the  measurement  system  that 
they  can  use  to  make  the  advances  that  they  want. 

So  we  obsessively  look  for  the  measurement  aspects  of  their  prob- 
lem and  the  industry  looks  for  the  technology,  the  product  develop- 
ment, all  of  the  things  that  make  them  competitive.  All  we  are — 
not  all,  but  the  critical  role  we  are  playing  is  that  we  use  those  ex- 
ercises to  help  plan  our  metrology  program. 

Mr.  Tanner.  Like  I  said,  I  wish  this  Congress  could  hear  what 
you  all  have  had  to  say  and  others  who  come  here.  Maybe  we  ought 
to  use  the  industry  feedback,  the  industry  input  that  certainly  oc- 
curs at  NIST  on  some  other  agencies  of  the  Federal  Government. 
People  have  this  uncontrollable  desire  to  attack  sometimes,  and  I 
understand  it,  and  the  government  gives  them  plenty  of  ammuni- 
tion with  which  to  fire.  But  this  is  a  critically  important  area  for 
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this  country  and  in  the  future  and  your  contribution  here  today 
and  what  you've  done  in  the  past  is  appreciated.  Thank  you. 

Mr.  GUTKNECHT.  Thank  you,  Mr.  Tanner. 

I  have  a  couple  of  questions  for  Dr.  Glaze.  You  talked  about 
Technology  Roadmaps.  I  understand  they're  prospective,  but  are 
there  any  potential  drawbacks  to  these  Roadmaps  in  other  poten- 
tial industries? 

Dr.  Glaze.  Drawbacks  to  other  industries? 

Mr.  GuTKNECHT.  Right.  I  mean,  are  Roadmaps  always  a  good 
idea,  the  concept? 

Dr.  Glaze.  I  think  we've  found  them  to  be  essential  for  our  in- 
dustry. I  can't  imagine  any  drawbacks  in  the  process  that  we've 
used,  and  in  view  of  the  fact  that  many  other  industries  are  now 
emulating  that  process — there  has  been  criticism,  minor  criticism 
by  some  that  says,  by  doing  a  Roadmap  exercise,  you  structure  the 
future  and  perhaps  stifle  creativity.  I  think  that  particular  concern 
is  pretty  baseless. 

In  doing  our  Roadmap  activities,  we  do  not  specify  solutions.  We 
only  specify  what  the  needs  are  going  to  be  to  meet  certain  mile- 
stones to  continue  the  productivity  increases  that  this  industry  has 
provided.  But  we  don't  say  that  it  must  be  achieved  by  this  specific 
method.  So  we  lay  out  primarily  the  requirements  and  the  needs 
as  opposed  to  the  methods. 

Mr.  GUTKNECHT.  Can  a  company  gain  competitive  advantage  and 
still  stay  on  the  Roadmap? 

Dr.  Glaze.  The  work  that  the  Roadmap  addresses  is  what  we 
call  pre-competitive.  In  other  words,  it  benefits  all  companies.  One 
way  to  look  at  it,  I  think,  is  that  the  massive  amount  of  R&D  that 
industry  spends,  much  of  it  is  spent  with  the  target  of  deplojrment 
within  three  years.  In  other  words,  industry  is  looking  three  years 
out,  sometimes  a  little  bit  further  than  that  out. 

If  you  get  much  past  three  years,  some  three  to  six  years — for  ex- 
ample, we  have  created  SEMATECH,  the  sort  of  help  on  a  pre-com- 
petitive basis  in  a  consortium  manner,  a  common  way  of  solving 
problems  in  the  three-  to  six-year  time  frame.  We  look  at  university 
research,  for  example,  as  providing  innovative  solutions  in  the 
nine-year  time  frame  and  out,  say,  for  example. 

So  if  you  look  at  the  very,  very  broad  spectrum  of  research  needs, 
there's  a  very  specific  role  for  both  in-house  competitive  research 
that  companies  will  use  to  gain  a  competitive  advantage  and  the 
pre-competitive  research  for  which  no  single  company  can  execute 
but  which  all  companies  within  this  pond,  this  industry,  require. 

I  don't  know  if  I  answered  the  question.  The  answer  is  that  to 
remain  competitive,  my  feeling,  you're  going  to  have  to  participate 
at  the  pre-competitive  level  in  this  technology  because  as  a  single 
company,  you  won't  be  able  to  do  it  yourself. 

Mr.  GUTKNECHT.  Dr.  Glaze,  I  want  to  thank  you  for  speaking  in 
plain  English.  You've  been  helpful.  I'm  still  not  sure  that  I  com- 
pletely understand  this  concept,  but  I  think  I  understand  it  better 
than  when  you  started. 

Dr.  Glaze.  The  concept  of  pre-competitive? 

Mr.  GUTKNECHT.  Right. 

Dr.  Glaze.  For  example 

Mr.  GUTKNECHT.  Yes,  give  an  example,  if  you  could. 
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Dr.  Glaze.  The  example  is  that,  basically,  the  design  rules,  for 
example,  in  laying  out  the  transistors  on  a  piece  of  silicon,  the  tool- 
ing of  the  metrology,  the  chemicals,  and  the  processes  are  in  com- 
mon use  throughout  the  industry.  You  do  not  go  to  Intel  and  find 
that  they  are  using  one  unique  set  of  lithography  tools  or  one 
unique  set  of  chemicals  or  one  unique  set  of  processes.  They  com- 
pete on  design,  performance.  But  the  suppliers  in  our  industry 
carry  a  similar  technology  base  out  to  the  entire  industry,  and  it's 
those  elements  of  the  technology  that  are  common  to  all  and  that's 
what  we  mean  by  pre-competitive. 

Mr.  GUTKNECHT.  I  think  I  understand  it  now.  Thank  you. 

Dr.  Glaze.  Okay. 

Mr.  GUTKNECHT.  Dr.  Rhyne,  you  mentioned  that  the  major  prob- 
lem was  a  mismatch  of  funds  and  needs,  and  you  used  the  term 
technology  triage.  Can  you  expand  a  little  on  that  and  how  big  a 
problem  it  is? 

Dr.  Rhyne.  The  thing  that  we've  seen  repeatedly  when  we've  sat 
down  with  the  managers  of  the  projects  at  maybe  the  second-tier 
level  within  the  EEEL,  moving  away  from  Jud,  away  from  the  divi- 
sion managers,  and  we've  had  people  sit  down  with  the  specific 
project  managers,  is  that  they  have  lots  and  lots  of  project  ideas, 
ideas  that  each  and  every  one  considered,  whether  from  a  social 
point  of  view,  a  technical  point  of  view,  whether  it  serves  industry 
or  what,  looked  very  good.  But  the  resources  consistently  are  inad- 
equate to  meet  this  ever-expanding  area  of  electrotechnology. 

We  use  terms  like  DC-to-daylight.  I  mean,  you  are  talking  about 
such  a  range  of  industries.  The  electric  utility  guys  think  that  61 
hertz  is  high  frequency,  and  the  guys  over  here  building  computers 
think  that  five  volts  is  high  voltage.  So  it's  just  a  tremendous  span 
of  work  to  be  done  and  each  of  these  industries  is  finding  that  the 
metrology  techniques  that  they  used  up  until  now  are  no  longer 
available  and  appropriate. 

Dr.  Hebner  mentioned  the  common  meter  in  the  home.  I  mean, 
there's  a  device  that  you  manufacture  for  a  very  small  amount  of 
money,  I  think  less  than  $10,  probably,  in  current  dollars,  and  yet 
it  works  without  problems  for  decades,  if  not  for  100  years.  But  the 
way  in  which  people  are  using  electric  utility  consumption  right 
now  is  arguing  that  some  differences  need  to  be  provided  in  the 
way  we  measure  how  much  you  actually  use.  What  exactly  are  we 
going  to  count  as  a  kilowatt  hour  when  you're  using  only  a  portion, 
maybe,  of  the  60-cycle  wave  form? 

All  of  these  things  are  being  thrown  on  the  table  and  what  the 
managers  are  trying  to  do  is  to  listen  to  the  industry  requirements 
and  respond  by  starting  projects  that,  albeit  maybe  in  a  three  to 
five  year  lifetime,  not  a  20-year  lifetime,  but  those  projects  have  to 
prepare  the  metrology  capabilities  so  that  U.S.  industries  will  be 
able  to  use  those  capabilities  to  produce  products  not  only  nation- 
ally but  internationally  at  some  point  in  the  future. 

Having  watched  the  process  and  watched  the  projects  that  get 
stopped  and  the  new  projects  that  do  get  started,  there's  so  much 
film  left  on  the  cutting  room  floor,  if  you  want  an  analogy.  There's 
just  a  lot  of  good  stuff  that  I'm  sure  they  would  like  to  do.  It's  easy 
to — there  are  very  few  problems  you  can't  solve  with  more  time, 
money,  or  people.  I'm  not  here  to  try  to  tell  you  that  you  need  to — 
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you've  got  a  lot  of  problems  that  are  hard  to  meet  with  the  monies 
available.  I  sit  on  a  local  school  board  in  Texas,  and  so  I  deal  with 
this  in  a  miniature  way. 

But  I  will  tell  you  that  with  what  they've  got,  the  thing  that  we 
are  always  impressed  with  is  how  well  they  are  matching  their  ca- 
pability, their  resources  to  those  needs,  but  it  is  inevitably  a  deci- 
sion making  process  that  leaves  good  ideas  behind,  and  I  think 
what  you  and  I  ought  to  be  worried  about,  I'm  sure  what  their 
management  worries  about  all  the  time,  is  are  they  going  to  miss 
something  today  that  we  really  wish  had  been  started  in  a  few 
years. 

That's  why  I  said  earlier  that  in  those  areas  where  the  industry 
is  better  organized,  as  in  Jim's  area,  which  is  actually  my  own 
area,  that  there  we're  getting  the  right  kind  of  support.  It's  in  the 
areas  that  the  industries  are  somewhat  disparate,  not  well  formed 
together,  that  I  think  all  of  us  worry  that  the/re  not  going  to  iden- 
tify the  future  metrology  needs  and  we  may  miss  some  critical  area 
that  will  be  of  major  national  importance  in  the  future. 

Mr.  GuTKNECHT.  I  want  to  thank  you.  We  sort  of  perform  triage 
around  here,  and  we  call  it  prioritization  sometimes,  but  I'm  cer- 
tain that  your  concerns  are  well  founded.  I'm  amazed  myself  at 
how  well  you  guys  do  in  terms  of  keeping  ahead  of  a  constantly- 
changing  world  of  technology  out  there. 

I  would  say,  though,  for  the  record,  just  looking  at  the  raw  num- 
bers in  terms  of  the  appropriations,  I  think  the  bill  that  Bob  Walk- 
er put  together  and  took  to  the  House  floor  and  hopefully  is  going 
to  be  negotiated  with  the  Senate  does  call  for  increasing  spending 
in  all  the  categories,  or  at  least  spending  as  much  as  the  President 
had  requested,  and  in  many  categories  we're  actually  increasing  it. 
But  there's  always  unlimited  demands,  particularly  in  this  field, 
and  we  do  congratulate  the  folks  at  NIST  and  the  work  that  they 
do. 

I  think.  Dr.  Hebner,  your  analogy  was  a  good  one,  and  I  think 
we  as  Members  of  Congress  should  expropriate  that  and  use  it, 
that  there  are  really  two  fundamental  responsibilities  of  the  gov- 
ernment, and  that  is  to  create  a  medium  of  exchange,  some  kind 
of  currency,  and  then  secondly,  some  system  of  weights  and  meas- 
ures, if  you  want  to  get  down  to  the  basics.  But  as  we  go  forward 
and  as  those  weights  and  measures  get  more  and  more  com- 
plicated, I  do  congratulate  the  work  that  you  do  to  try  and  stay 
ahead  of  that  in  this  field. 

I  have  no  further  questions.  Unfortunately,  the  Chairwoman, 
Mrs.  Morella,  has  not  returned.  If  there  are  no  other  further  ques- 
tions or  business  to  come  before  the  Subcommittee,  I  will  call  this 
Subcommittee  meeting  adjourned.  Thank  you. 

[Whereupon,  at  2:44  p.m.,  the  subcommittee  was  adjourned.] 
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